Proposed New Work - Some comments

Brian Wichmann

1  Introduction

2  Issues

Title:

Guidance for Avoiding Vulnerabilities in Programming Languages sounds a bit narrow to me. Surely we are aiming at higher quality software which is rather broader. For instance, the use of annotations or assertions which might not be formally part of the language would seem to me to be important.

Also, in my mind, the word Vulnerabilities is associated with security rather than safety. I think both are important and need to be addressed (as the rest of the NWI proposal makes clear). Not surprisingly, I favour the title of [1].

Tools:

I think one key to the whole area is the development of very high quality tools which analysis source code. There are a lot of these for C but it is very hard to see exactly what their impact is upon code quality after their application. Tool qualification in this are is extraordinarily difficult.

Hence I think the NWI needs to be strengthened in suggesting that tool qualification should be made more tractable by the TR providing quantifiable goals.

Language selection:

We need to be very careful here since it seems that language selection is often made on non-technical grounds. Strong statements could result in the entire TR being ignored!

Predictable behaviour:

As Derek points out, this is a complex topic and deserves proper study. Even if only this issue could be properly handled, I would count the TR a success. In my view, this aspect needs to be added to the NWI proposal.

Derek states correctly: A translator is likely, but not required, to generate the same machine code from given source if the same set of options are used. I regard that as unacceptable for a translator adhering to the proposed guidelines. (But surely the title does need broadening?) My understanding was that a compiler (no longer in use, I think) did more optimisation with more store, but the amount of store the compiler had was determined by the workload on the machine and hence not controlled by a user option.

Tractable analysis:

As Derek points out, some forms of analysis can be very hard. In many cases, annotations can make a big difference.

Analysis not feasible:

We need to point out the risks inherent when an analysis is not feasible. This impacts language selection and the potential use of annotations.

2.1  Static analysis

2.2  Correctness versus Probability

Correct by construction:

This approach aims high and may well (but is not required) to use Formal Methods. The whole design is aimed at zero defects. It clearly requires a very precise specification.

Validation and verification of such systems would also be undertaken with the approach of aiming at zero defects.

Reducing the probability of errors:

Given perhaps a conventional development of software, careful analysis and use of tools can help detect errors which can then be removed. Evidence from failures or specific bug reports can also be used to remove errors. Reliability growth models can then be used to quantify the residual errors.

Regression testing of compilers seems to be particularly effective in removing errors - compilers are typically quite reliable in spite of their complexity.

References

[1]

TR ISO/IEC 15942:2000, Guide for the use of the Ada programming language in high integrity systems.

A  Document details

1. First written 26th May 2005.