SEBoK Review and Adjudication

From SEBoK
Revision as of 20:10, 17 April 2013 by Bkcase (talk | contribs)
Jump to: navigation, search

The participation of the worldwide systems engineering (SE) community in the open collaboration process played a significant role in shaping version 1.0 of the Guide to the Systems Engineering Body of Knowledge (SEBoK). The SEBoK editors made a handful of additional changes to the SEBoK to create version 1.0.1 (see Release History). Those changes were so minor (e.g., correcting spelling errors) that they did not undergo additional public review.

This article outlines the process followed to solicit and adjudicate review comments received on the developmental versions of the SEBoK (i.e. versions 0.25, 0.5 and 0.75). A comprehensive list of all comments received and the authors’ response to those comments is also included in adjudication matrices.

Reviewers and Review Comments

SEBoK v. 0.25

The first version of the SEBoK, version 0.25, was released as a document for limited review in September 2010. A list of potential reviewers was compiled based on

  • direct requests to professional societies, corporations, and academic institutions; and
  • volunteers who responded to requests during SEBoK related presentations and talks at conferences and other events.

The SEBoK v. 0.25 document, along with a review form with over thirty questions and detailed instructions were distributed to the potential reviewers.

Reviews were received from 114 reviewers who provided over 3000 comments by December 2010.

SEBoK v. 0.5

A number of changes in the structure and content of the SEBoK was made in response to the review comments received for SEBoK v. 0.25. The next version of the SEBoK, version 0.5, was released in September 2011, as a wiki. In addition to providing easy access and navigation of the SEBoK, the wiki platform also enabled broader review by the worldwide systems engineering community. Release information and requests for review were sent out to an updated list of reviewers. Review comments on individual articles were predominantly received on the wiki. A review form was also distributed to receive overarching comments on the SEBoK as a whole and on the wiki implementation. Comments were also received in various other formats including emails, spreadsheets and marked-up documents until December 2011.

SEBoK v. 0.75

Though not originally planned, the third interim version of the SEBoK, version 0.75, was released in March 2012 for a short limited period review until April 2012. Not all of the review comments were addressed in SEBoK v. 0.75. The wiki platform was well received by the worldwide SE community and so was retained for future releases of the SEBoK. More features for reviewing and navigating were added to the SEBoK wiki. The wiki remained open for review, though it was stated that the adjudication of those comments may not be addressed for SEBoK v. 1.0.

SEBoK v. 1.0

The wiki continues to remain open to review comments as SEBoK v. 1.0 is released and is now available worldwide. Comments received here, and those that were received after the review period of SEBoK v. 0.75 was closed would be adjudicated for future versions of the SEBoK.

Adjudication Process

A thorough process involving the authors and Part Team Leads (PTLs) was used to address every review comment received on the SEBoK. This process largely remained the same for all versions of the SEBoK that were released.

Once the review period for a SEBoK version was over, the following broad steps were followed:

  1. All comments received in various formats were all compiled in matrices as spreadsheets, segregated by SEBoK parts. When warranted, compound comments were broken down into individual comments to support effective response to the comments.
  2. The comments were then distributed to the respective PTLs and authors
  3. The authors responded to every review comment by choosing one of the adjudication actions listed in Table 1 below. Where required, authors provided reasons and rationale for their decision on particular review comments.
  4. Adjudication matrices were then re-compiled incorporating author responses as spreadsheets for further use and distribution.

Table 1. Types of Adjudication Actions. (SEBoK Original)
Action Description
  • The comment from the reviewer has been accepted in whole, and suggested action has been taken.
  • The comment from the reviewer has been accepted in part and suggested action has been taken with some modification(s).
  • Details of the modification is provided.
  • Suggestion recommended by the reviewer has been declined.
  • Rationale for this decision is provided
  • The comment made by the reviewer requires no further action.
  • This comment is similar to a comment already received and adjudicated (in this or earlier version of the SEBoK).
  • Response to the comment is deferred to a future version of the SEBoK.

Adjudication Matrices

A detailed adjudication report was prepared based on the reviews received on SEBoK v. 0.25. For the review comments on the 0.5 and 0.75 versions of the SEBoK, the adjudication matrices have been prepared. These may be downloaded (in PDF format) from the following links:

It may be noted that some review comments may be relevant only to the SEBoK version on which the comments were made; content may have changed since then. Names of individual reviewers have not been included in any of the above adjudication matrices. If you reviewed the an earlier version of the SEBoK and would like to know your reviewer number and see the responses to your comments, please contact

SEBoK v. 1.9.1 released 30 September 2018

SEBoK Discussion

Please provide your comments and feedback on the SEBoK below. You will need to log in to DISQUS using an existing account (e.g. Yahoo, Google, Facebook, Twitter, etc.) or create a DISQUS account. Simply type your comment in the text field below and DISQUS will guide you through the login or registration steps. Feedback will be archived and used for future updates to the SEBoK. If you provided a comment that is no longer listed, that comment has been adjudicated. You can view adjudication for comments submitted prior to SEBoK v. 1.0 at SEBoK Review and Adjudication. Later comments are addressed and changes are summarized in the Letter from the Editor and Acknowledgements and Release History.

If you would like to provide edits on this article, recommend new content, or make comments on the SEBoK as a whole, please see the SEBoK Sandbox.

blog comments powered by Disqus