A quick theory of system complexity (federicopereiro.com)

A new theory on system complexity has been proposed, outlining how complexity impacts the efficiency of software development and system modifications. The theory establishes three key rules: the effort required for changes correlates with the size of the change and the system’s complexity; complexity is defined by the effort needed to understand the system; and complexity grows exponentially relative to a team’s understanding and the effort applied. This highlights that as systems become more complex, the required effort to comprehend and manage them increases, potentially overwhelming even large, skilled teams. The significance of this theory for the AI/ML community lies in its implications for system design and maintenance. As applications and AI models grow in complexity, the ability to efficiently add features or make changes diminishes unless teams simplify their systems. The theory suggests that minimizing complexity is crucial for long-term speed and efficiency in development. Additionally, a proposed correlation between the number of lines of code and complexity hints at underlying quadratic relationships, similar to those seen in large language model training. This underscores the need for clear, simplified representations of complex systems to maintain agility and effectiveness throughout their lifecycle.