Transparency, CODIL, and the Horizon "black box" Software

 

One of the biggest difficulties faced by chatbots and other AI packages based on large language models is that they are black box systems which lack transparency. The same difficulty can also apply to large conventionally programmed commercial applications where the lack of transparency can have serious consequences. This problem has been spectacularly demonstrated by the problems caused by the General Post Office's Horizon software where innocent postmasters ended in prison because of errors in the "black box" software. Part of the problem was that the courts were prepared to accept as reliable "evidence" from a complex "black box" system which was unable to answer questions about the origins or reliability of the evidence it was being provided.

So why was the Horizon software not written in a transparent way? The software was produced by what had been the part of Fujitsu which had formerly been the UK company International Computers Ltd (ICL). ICL had been formed in 1966 by a government-inspired merger of a number of companies including English Electric LEO Marconi. Shortly after ICL was formed the company board decided (after much argument) to build a next generation computer (the 2900 series).  Money was short and the decision was made to close down research projects (particularly those involving hardware) unless they were directly involved in the plans dor the proposed 2900 series of computers.

 

One of the closed down projects was the CODIL project - which was concerned with building a transparent human interface for use  in large commercial management information systems. The language was based on a study of the one of the most complex sales accounting system (the LEO III computers at Shell-Mex and BP) and as far as I know it was the only research which directly addressed the need for transparency in large and complex commercial organisation. 

It was clear that John Pinkerton did not want the project to close and he made it possible to take the research (but no funding) to a university. H also helped me to write  paper which was accepted by the Computer Journal - but  the paper had to concentrate on the algorithms as it could embarrass ICL if the paper revealed the company was not interested in pursuing the possible commercial advantages.

 

There was one interesting event on virtually my last day working for ICL. One other person working on the design of computer CPU (previously for ICT) was also leaving and we were both asked attend a seminar on CPU design at a Ferranti research laboratory. The general reaction to my presentation was that virtually all requests for better CPUs were based on the requirements of professional programmers and my approach was the first time someone had suggested how a CPU could be made more human-friendly for non-programmers.  I very got the impression that if only  they had been told about my research months before it was unlikely that I would probably not have been made redundant.  A few days later (having left ICL) I presented my first conference paper on CODIL at the Conference on Man-Computer Interaction held at Teddington on 2-4 September, 1970 and I later learnt that I got my next job, working on the massive Linesman/Mediator project, as a result of this presentation.

 

What this story suggests is that about 15 years before ICL/Fujitsu implemented the disastrous "black box" Horizon project the Board of ICL had killed off research into building transparent computer systems, and this may have happened as a result of Board Room politics rather than any serious consideration of the research.

 

In more general terms the story also reveals the problems of trying to reconstruct the history of computing from inadequate project management records. When I went through the online index of the ICL archives the only references to the CODIL project I could find were in a slim A4 folder among John Pinkerton's papers - where most of the documents were also in the project archive that I hold. None of the reports to support the setting up of the project are covered, and a only a few of the relevant management records. In addition there is are no program listings, program specification, or example of the system under test. Basically the official ICL archive hold virtually nothing which allows you to reconstruct the project's history or to understand that the project was trying at replacing unreliable "black box" systems with transparent software which could explain what it was doing in human-friendly terms.

 

Once I realised this I browsed through the ICL archive index and it quickly became obvious that if a product had reached the stage of being marketed the archive contained major documents such as manuals and publicity information- but there seems to be very little about the way decisions were made about which projects were successful and which were abandoned. Maybe some of this information is preserved in reminiscences but in general it seems likely that there are few archives which relate to internal company politics underlying the decision which project advances to marketing and which project dies because of lack of support is lost.

 

If further properly refereed research shows that the CODIL pilot program written in 1968/9 was a working prototype model of how the brain processes information the "political" reasons why the project failed to get adequate support could be of significant interest. Even if not of such significant scientific interest, the CODIL archive forms a virtually complete record of how an interesting "blue sky" ideas, triggered by a neurodiverse individual, can be suppressed because the idea was too far "outside the establishment box."