Values in Design in the Age of Optimization: The Example of Privacy by Design

Values in Design in the Age of Optimization:

The Example of Privacy by Design

Seda Gurses, Technische Universiteit Delft

As controversial as it may sound, bringing in values into design is always also a matter of labor and cost, and not just a matter of ethical engineering practices. In the context of this presentation, we will focus on privacy as a value and raise questions about the cost of designing and producing privacy by design. We hope that this example will provide a way to explore greater questions around bringing values in design into software development as optimization becomes its key modus of production. Rarely do its advocates consider this cost when speaking of privacy by design. Many have argued that privacy engineering requires special skills, that privacy is not included in business models [0], or that it gets thrown under the bus because many companies must generate revenue from advertising [1,2]. Others have argued that privacy by design lacks engineering models[3,4,5,6] and, even when mandated through legislation, is underspecified from a technical standpoint [7]. While all of these points make sense, what we also need to focus on is perhaps the simplest: privacy by design requires labor and resources, which are economic expenditures. You have to pay for it. Intuitively, this all sounds like a straightforward matter of cost accounting: to have privacy concerns or data protection requirements reflected in the design of a system means extra hours of labor, and maybe some extra features, and therefore more money is required to pay the salaries of the people who engineer products with privacy in mind. While this is an obvious concern, framed in this way, cost is solely an issue of accounting and resource management. This paper takes a much deeper dive into the issue of cost, risks and its relationship to the greater business imperative, to show how deeply it shapes the production of software. The way software engineering is shaped has a direct impact on what privacy issues arise and can be addressed. Looking at the problem from the perspective of software engineers, this paper argues that the software industry’s aspirations to maximize profit while minimizing expenditures manifests itself in the very choices that get made in how technology will be produced. Cost control plays a role in the engineering methodologies companies have promoted as well as in the tools and infrastructure they provide or which their developers choose to adopt [,13]. As a result, even when skilled engineers are available to do the work, even when business models accommodate privacy concerns, even when the objective of regulation has not been obscured by the process of multi-stakeholder drafting, from the perspective of software engineering, privacy by design remains an elusive project. As the paper will show, because of cost control, the way software production is organized is antithetical to addressing privacy through technical means. The contention of this talk is that the organization of software production around costs, value creation [14] and risks [15,16,17] makes a difference in our capacity to produce privacy-minded technologies . Organized as it is now, production processes either constrain the benefits of applying privacy methods, increases the complexity of doing so, or requires changes to engineering practice that are so costly to implement they are unlikely to be adopted. The empirical story we will tell is a part of the much larger history of how the economic imperatives that produce software, in a cost saving and value maximizing, has already shaped the software industry’s methodologies and products. There are many mechanisms through which software companies secure their profits and reduce their costs, e.g., software patents, different licensing and investment models [18]. What is of direct interest to us is the way in which software engineering has been transformed to fit economic outcomes, and how AI promises to make it even more so. While many different mechanisms have a role in what we are studying, we specifically set out to show how the search for effective software production and ‘business agility’ has led to the popularity of ‘agile development methodologies’, ‘service architectures’, ‘lean products’ and ‘cloud infrastructures’. In combination, these different methods and technical architectures have come to cement the use of data feedbacks in the service of managerial forms as well as technical forms of optimization in the production of software and extraction of value [19]. The empirical results that this paper builds on point to the way in which software production is infatuated with labor costs specifically, and cost sinking in general. While there are degrees to which such cost reduction can be applied, a good amount is deeply entrenched in practices and infrastructures: agile teams, service architectures, and cloud infrastructures organized under the rule of objective functions. Privacy-by-design, to be successful, requires some of these practices to be undone. It requires both giving up on cost-saving development practices and software infrastructures, and requires practically rethinking how we produce software [20]. Seen this way, privacy by design surfaces that its realization requires greater political and economic discussion around how software itself is brought to the world.   [0] Rubinstein, Ira, Big Data: The End of Privacy or a New Beginning? (October 5, 2012). International Data Privacy Law (2013 Forthcoming); NYU School of Law, Public Law Research Paper No. 12-56. Available at SSRN:  or [1] Hoofnagle, Chris Jay and Urban, Jennifer M. and Li, Su, Privacy and Modern Advertising: Most US Internet Users Want 'Do Not Track' to Stop Collection of Data about their Online Activities (October 8, 2012). Amsterdam Privacy Conference, 2012. Available at SSRN: [2] Toubiana, Vincent and Narayanan, Arvind and Boneh, Dan and Nissenbaum, Helen F. and Barocas, Solon, Adnostic: Privacy Preserving Targeted Advertising (2010). Proceedings Network and Distributed System Symposium, March 2010. Available at SSRN: [3] Shapiro, Stuart S. "Privacy Risk Analysis Based on System Control Structures: Adapting System-Theoretic Process Analysis for Privacy Engineering," 2016 IEEE Security and Privacy Workshops (SPW), San Jose, CA, 2016, pp. 17-24. [4] Brooks, S., Brooks, S., Garcia, M., Lefkovitz, N., Lightman, S., & Nadeau, E. (2017). An introduction to privacy engineering and risk management in federal systems (pp. 1-49). US Department of Commerce, National Institute of Standards and Technology. [5] Martin, Y. and Kung, A. "Methods and Tools for GDPR Compliance Through Privacy and Data Protection Engineering," 2018 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), London, 2018, pp. 108-111. [6] Gürses, Seda and del Alamo, Jose, "Privacy Engineering: Shaping an Emerging Field of Research and Practice," in IEEE Security & Privacy, vol. 14, no. 2, pp. 40-46, Mar.-Apr. 2016. [7] Bygrave, Lee A. "Data protection by design and by default: deciphering the EU’s legislative requirements." Oslo Law Review 4, no. 02 (2017): 105-120. [8] OWASP, Secure Software Contract Annex, , Checked on February 2020. [9] IBM Software Services, Improving Software Economics, Rational: Top 10 Principles of Achieving Agility at Scale, 2009, [10] Cantor, M. and Royce, W., "Economic Governance of Software Delivery," in IEEE Software, vol. 31, no. 1, pp. 54-61, Jan.-Feb. 2014. [11] Fox, Armando, Rean Griffith, Anthony Joseph, Randy Katz, Andrew Konwinski, Gunho Lee, D. Patterson, Ariel Rabkin, and Ion Stoica. "Above the clouds: A berkeley view of cloud computing." Dept. Electrical Eng. and Comput. Sciences, University of California, Berkeley, Rep. UCB/EECS 28, no.13 (2009): 2009. [12] B. C. Tak, B. Urgaonkar and A. Sivasubramaniam, "Cloudy with a Chance of Cost Savings," in IEEE Transactions on Parallel and Distributed Systems, vol. 24, no. 6, pp. 1223-1233, June 2013. [13] Newcomer, Eric, and Greg Lomow. Understanding SOA with Web services. Addison-Wesley, 2005. [14] Hartmann, Deborah, and Robin Dymond. "Appropriate agile measurement: using metrics and diagnostics to deliver business value." In AGILE 2006 (AGILE'06), pp. 6-pp. IEEE, 2006. [15] Gentzoglanis, Anastassios. "Risk, financial modeling and cloud computing: a new approach." Computer 9 (2011): 147-151. [16] ITIL – Information Technology Infrastructure Library. Technical report, OGC, 2011. [17] Maglio, Paul P., and Jim Spohrer. "Fundamentals of service science." Journal of the academy of marketing science 36, no. 1 (2008): 18-20. [18] Cohen, Julie E. "Between truth and power." In Information, Freedom and Property, pp. 69-92. Routledge, 2016. [19] Kulynych, Bogdan, Rebekah Overdorf, Carmela Troncoso, and Seda Gürses. "POTs: protective optimization technologies." In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 177-188. 2020. [20] Kostova, Blagovesta, Gürses, Seda, Troncoso, Carmela, On the challenges of deploying Privacy Enhancing Technologies (PETs), (under submission)