Software Engineering Outside the Box

"Unboxing responsible requirements engineering"

In December 2023, I attended the ESEC/FSE conference in San Francisco, and in addition to hearing about lots of interesting work relating to human aspects of software engineering, and security, I also enjoyed listening to the keynote talks delivered at the event. I already shared some reflections on one of these keynotes in a previous blog post (GenAI for Software Engineering) and in this one I cover my notes on another keynote which was delivered by Professor Margaret Burnett, a University Distinguished Professor from Oregon State. 

Her talk was titled "Getting outside the Bug Boxes" and highlighted the importance of distinguishing between the goals (or policies) of software engineering from the mechanisms (or tools and methods) used to engineer software. The central thesis was that, too often, software engineers constrain themselves inside a box that is based on the latter and therefore miss important aspects of the former. To expand on this, 4 different types of "box" were discussed.

Box 1 | Mechanism boxes: these boxes are where assumptions are made about the effect of providing users with tools (or mechanisms) without thinking carefully about the users' goals. The talk highlighted the example of end-user programming, focussing on spreadsheets. Here, the versatility of the environment, allowed users to define arbitrary data tables and perform computations on them using formulae and additional mechanisms allowed them to automate the manipulation of the formulae when copying them across cells, etc.  

Professor Burnett's research on end-user programming experiences in these environments highlighted challenges with this approach including, spreadsheets full of bugs, bad programs leading to bad decisions, and overconfidence of users. The mechanism-based thinking that is promoted by spreadsheet-based end-user programming promotes a blind application of tools.

A better approach, according to Professor Burnett, would be to consider the end user programmers' actual goal (or policy), which was to: "Develop a computational tool (software) that is reliable enough for my purpose". This would lead to designing spreadsheet software the allowed the end user programmers to define their reliability and purpose requirements, and highlight how well the data tables and formulae they were specifying met these requirements.

Box 2 | Life-cycle boxes: these boxes are where we think about the software development life-cycle as being only linear or iterative. As a result, the correctness of the software depends on how well the requirements were understood at the relevant life-cycle stage and bugs introduced at this stage get 'baked in'. 

Some key points made in this section of the talk included the need to consider 'end-user intent' when assessing bugs and working on fixes for them. Techniques such as 5-whys, to get to the bottom of why a user is reporting an issue could help the developers understand the root problem and fix this, rather than implementing a superficial fix.

Box 3 | Putting the end-user in a box: these boxes relate to the assumptions that developers make about end users of the software and the need for techniques that support challenging and/or validating these assumptions. The example used by Professor Burnett was gender-inclusivity bugs where assumptions about the gender of users lead to omissions in the design of software or the fixing of bugs that are reported. The talk highlighted the use of the Gender Mag (Gender Magnifier) technique (Burnett et al., 2016) to surface these issues and make sure they are addressed by software engineers.

There is a balance to be struck between grouping users together to take advantage of economies of scale when addressing software defects vs. recognising the individual needs of users. Some of the work we have been doing at the Open University and Lero on integrating social identity theory into software engineering - whether for adaptive privacy (Calikli et al., 2016) or collaboration (Kordoni et al., 2023)

Box 4 | The bug box: this final type of bug focuses on the assumption that software defects are only present in the code. The key point here is to recognise that bugs can arise in other artefacts, including requirements, design, configurations, and deployments of software. It highlights the need for debugging activities to think beyond the code, identify the source of a bug and address the root cause. It also offers the possibility for automated debugging techniques to draw on a wider range of knowledge to provide better explanations of debugging - i.e. explanatory debugging.

From my perspective, Prof. Burnett's main call to action was one that called on us to make requirements engineering a more central part of software engineering practice, ensuring that we understand the problem before trying to use computing to solve or automate something. I would go even further to say that we need to embed a practice of responsible requirements engineering. Here we should be willing to challenge the assumption that the problem even requires a technological solution and instead take a more holistic systems approach to identifying and implementing the most appropriate solution - which may be outside the software box.

I look forward to exploring some of these issues and related solutions in the context of the recently funded Centre for Protecting Women Online, which includes a key strand on responsible software engineering.

References:

Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., Peters, A. and Jernigan, W., 2016. GenderMag: A method for evaluating software's gender inclusiveness. Interacting with Computers, 28(6), pp.760-787.

Calikli, G., Law, M., Bandara, A.K., Russo, A., Dickens, L., Price, B.A., Stuart, A., Levine, M. and Nuseibeh, B., 2016, May. Privacy dynamics: Learning privacy norms for social software. In Proceedings of the 11th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (pp. 47-56).

Kordoni, A., Gavidia-Calderon, C. and Nuseibeh, B., 2023. “Are we in this together?”: embedding social identity detection in drones improves emergency coordination. Frontiers in Psychology, 14, p.1146056.

Image: "Software engineering outside the box" Generated using DALL-E. A whimsical, futuristic setting where software engineers are depicted not just sitting in front of computers, but also engaging with technology in imaginative ways. Depicted as a scene with a diverse group of engineers standing around a large, glowing, holographic display of code, interacting with it using gestures. Around them, floating digital screens display various projects, from virtual reality landscapes to AI robots.