# efficiency: focusing on efficiency leads to complexity.

it is very easy for me to fall into the trap of obsessing about efficiency. efficiency can mean many things: in software i might be optimizing for minimal ram usage, minimal time use, or in real world i might be optimizing to spend as little money as possible. however achieving peak efficiency has downsides. the principle of pareto efficiency suggests that the more efficient i make something, i lose optimality in other dimensions. the principle introduces the concept of pareto frontier: when i am on the frontier any improvement in my preferred dimension will result in decline in some other dimension. examples:

• i want my software to use very little memory. but to achieve this i would need to use very complex data structures and thus maintaing the software would be a nightmare.
• i am a baker and i want to bake bread as cheap as possible. therefore i will look for the cheapest flour possible. but such flour makes my bread taste and smell bad. i compensate by adding even more cheap additives. now i am not selling bread anymore just a bread-like hodgepodge.
• i want to commute to work in minimal time possible. so i take a bike instead of walking. but now i have to bike next to the traffic and as such it is super stressful. i am exposing myself to a lot of stress in exchange of less time.

i am not saying that i should embrace inefficiency but rather just to avoid the obsession to be at the frontier. it is much easier to adapt to the changing world when i have a lot of wiggle room. this leads me to the next question: why the obsession about the efficiency?

suppose i am a big company. i make 100 million dollars yearly. if i hire a person that can make 1% improvement to my processes then that means i can now make 101 million dollars yearly. that is 1 million dollars worth of extra money in exchange for one person's salary. sure, it adds complexity to my processes but that person can deal with it so it is a sound tradeoff.

other examples: i am a farmer and improving crop efficiency means i can feed more people. or improving fuel efficiency means less pollution. in many the other cases like these the benefits do outweigh the downsides. but it is important to be aware of those downsides.

the common theme i see is that efficiency starts to matter mostly in large systems where even the smallest improvements lead to large, quantifiable differences. the same system can also afford the kafkaesque complexity stemming from all the small efficiency "improvements". but when other people try to use or rely on the output of that system then they are inadvertently exposed to the risks stemming from all that complexity.

• in the software's case i would get giant, unmaintainable codebases. i would be discouraged to even attempt fixing bugs in software because i do not know where to even start or my machine is not powerful to build the software in question.
• in the crops case i might get more confident about my farming abilities and start growing a single plant with great efficiency. the risk here is that a lot of people might start depending on me solely. if some unforeseen problem arises with my plant and i lose a year's worth of crop that will expose all those people to huge amount of risk. in the case where i am forced to keep a large variety of plants with various successes it would keep people's expectations in check.
• or take the fuel efficiency example: it means that more people can afford personal cars, less pollution means less people will be advocating against cars, which means there is the chance that there will be a demand for even more cars and in total there will be more pollution by making all those cars.

although i enjoy the efficiencies of the modern world but when it comes to my own projects, i stopped caring about efficiency and optimality. my new focus is to not let things grow unbounded. i set arbitrary limits on myself or on the software i write and i adhere to them. when i reach a limit somewhere, i take a deep look to see if i can solve the problem in a different way so that i do not need to increase the limit. this approach lets me avoid being stuck in some local minima and reach much higher efficiencies in total. for example if i have a simple software that only solves 80% of my usecase, i will try changing my usecase rather than grabbing a more complex software. usually my environment gets simpler in exchange. for instance if all i want to have a blog with some text in it, i can just write simple text files rather than html or other complicated markup that needs various postprocessing before i can serve them. this way i do not need to deal with fonts, css, hyperlinks and whatnot. only the important bit, the text itself, remains. and most important observation from this is that when i avoid the unnecessary fluff, things get miraculously efficient. for instance this website loads quite fast without any effort on my side. no need to set up compression or use fancy protocols to make it fast.

i am pretty sure the efficiency stemming from limits extends to other cases too. if we did not have that much cars available, people would be more dependent on public transportation so the planet in total would be more efficient. or if less food would be available, we would have also less waste so probably we would try to be more conscious about food. of course we would have harder, less convenient life but potentially more efficient in exchange. this might not sound appealing for everyone but i think applying some limits on ourselves might get us some improvements and efficiencies in exchange without an explicit focus on efficiency itself.

published on 2018-04-27

to the frontpage