[TOS] FOSS Project Test and Requirements Management Opportunities

Joel Sherrill joel.sherrill at gmail.com
Sat Jun 12 15:15:03 UTC 2010


Hi,

I have lurked here since the beginning.  I am the maintainer of
RTEMS (http://www.rtems.org).  I apologize since this might
come against as a meandering email but there is a point in that
we have a couple of interesting areas that we could support
student involvement via either class projects or individual student
capstone projects.

The RTEMS Project has focused a lot over the past few years on
test coverage analysis.  We can now produce instruction executed
and branch taken/not reports for our test suite on 6 architectures
and have near 100% instruction coverage of core parts of the RTOS.

http://www.rtems.org/ftp/pub/rtems/people/joel/coverage/

We have had students for the past two GSOCs working on this
and these reports can be generated locally.  This year the student
is working on expanding the coverage into support areas.

OPPORTUNITY #1: Our reports are on the web and point to areas
in the code that have coverage issues.  The reports track it back
to the source.  We have the capability to associate explanations
with individual cases so we can analyse a group and hand them
to students.  This opportunity is a lot of small projects of varying
complexity.  Some are simply where there is not an test case
for a parameter error.  Others require more complex tests.

The focus on test coverage has lead us to investigate critical software
standards like DO-178B.  I don't think a FOSS project could by itself
certify (mainly because of resources) but I think that a FOSS project
could provide the technical data that would make it possible to
go through a certification.  So the focus is on improving our processes
and moving toward the general requirements of software validation
standards.

In this light, I have realized that although much of RTEMS is based
on standards like POSIX, RTEID, ORKID, and RFCs.  We cite
them frequently during implementation, testing, and bug resolution,
but we have no trail for tracking them from standard -> code -> test.

OPPORTUNITY #2: (Bigger projects).  We need a free requirements
tracking system that is suitable for an open distributed project.  We
need help in turning standards into requirements sets.  Tracking those
through the implementation and figuring our how to document which
test cases verify which requirements.  Ideally we would like to augment
our coverage analysis programs to also be able to say you ran the
test code for requirements X and Y but never tested code for
requirement Z.

So opportunity #1 is more coding and analysis.  Opportunity #2 is
a software engineering/management type project at the beginning to
establish a requirements tracking system for a distributed FOSS project
and help us integrate that into our processes.

The follow up to Opportunity #2 is helping translate those standards
documents into requirements.  POSIX 1003.1b (the Single UNIX Specification)
alone is quite large.

As you look forward to classes needing projects, students needing
capstone projects, Master's Theses, etc., please consider RTEMS
as a potential source of projects. [1]

Thanks.

--joel sherrill
RTEMS

[1] Many in the RTEMS community, including myself, have advanced
degrees and are capable of being outside advisers on graduate projects.



More information about the tos mailing list