Documentation and Testing Meeting 2010-08-11
Date: August 11, 2010
Janet Campbell, Arien Malec, Tony Calice, Keith Boone, Dragon Bashyam, Parag More More, Douglas Pratt, Karen Witting, Will Ross, Uvinie Hettiaratchy
||Due Date |
||Form plans to work with other workgroups for last ½ hour of “-a-thons” time on Tuesday morning of Face to Face, will touch base with Brian Behlendorf
||Will draft higher level policy guide and put out a call to people who want to get involved
||Write abstract for XDD developers’ guide
||Develop a list of documents the WG wants printed in advance of next week’s Face to Face meeting
||Will own Content Security for Simple Heath Transport. Will write abstract and start a wiki page.
||Will use the wiki page to start reviewing others’ work and offer comments and will use the mailing list if they want their documents reviewed.
- Review Documentation Priorities survey results
- Make sure everyone's on board with the artifacts that were voted N/A
- Review and confirm that Priority 1 artifacts are still Priority 1
- Assign volunteers for Priority 2 and Priority 3.
- Check in on progress for all documents currently under editing
- Plan for Docuthon at Face to Face
· Last week’s major goal was prioritizing all the documentation on the table.
· Mostly distribution of priorities was vague, with opinions split between priorities 2 and 3, so after some discussion we can shift things down.
· 3 documents raised a lot of questions: why are we doing this? What exactly is this?
§ For this meeting she wants to review why/if they are priorities.
· First document: SMTP Clients Development Guide, which includes use of reference implementation libraries.
§ Interesting because large range of opinions about this than the other two documents.
§ Could someone describe?
· Sees this doc as a more general item depending on how the client libraries come out. This is more than the API. It gives a reference guide, a how-to guide, how to install Java and complete basic tasks. If you are an EHR developer and want to take and download a library and put in source tree using API, this is the doc.
· So the client code needs to be well established before this can be written?
· Yes, sees this as a third priority, first needing the code.
· Next is the XDD Priority guide.
o Overall priority #2 by consensus.
· Fair statement. They are not yet ready to write the document.
· Objects. Feels we don’t have an understanding of where the code is.
· Sees the XDD document having two purposes:
o One, collect everything that is already written, and consolidate in one place.
o Two, find the need that the code would fill. Doesn’t understand why we need to wait for coders to act, unless it is decided that they will be making decisions instead of the documentation and testing or other groups.
· XDD specialization is separate, and is at priority 2.
· Retracts objection.
· But it’s true that the people writing clearly and the people figuring out the code need to come together to make sure ends meet.
· Last document marked inapplicable was the Testing Guide.
· Introduces certification/performance Guide.
· Should we retire the testing guide in favor of the conformance Guide?
· Conformance guide: tells you what to do.
· Testing guide: tests if something does what it says it will do. More about test scripts, documentation about test units.
· So in this context, conformance guide is meaningful use guide and test guide is test scripts.
· Are we also writing unit scripts?
· Yes, Ref Imp should write unit scripts.
· Testing guide is especially about end-to-end testing.
· Sees the line as getting blurry, as John had commented. Would like to know more about the distinction.
· Can we do that now?
· Thinks the conformance guide is more at the behavior level and these are the basic set of paths it has to perform. John pointed out that based on the deployment model, there should be a different level that should be reached in the guide.
· Deployment models=typologies.
· Testing for interactions.
· When describing results of tests, is that in conformance guide or testing guide?
· Conformance guide says which behaviors/functions need to be exhibited.
· Testing guide tells how to conduct a test if you want to test something.
· So, in other words, when sending a message from first to second point, this shall do this, this shall do that.
· In order to test, I could send a message from here to here, and it should be able to do x behavior.
· Yes, you could have conformance for something that can’t be tested.
· Thinks it could be valuable if we could develop the distinction with observable results. Wants it to provide guidance on appropriate use of HISPs.
· The test guide would show how well is it conforming to our objectives. Do we need the detailed steps or just the functional level things that we would be able to do with it?
· Yes, much more integrated and detailed steps not needed.
· By giving these inputs, one is proving the behavior.
· Seems testing guide grows out of the conformance guide.
· Yes, and both need to work hand in hand with the Reference Implementation WG.
o Make sure the Java code and the CSharp code are functionally identical.
o As we proceed should work in conjunction with Ref Imp WG to make sure we have sufficient proof the two functions do the same thing, that they do work end to end.
· Differs. Specification should influence the implementations.
· Final result should include interoperability testing process, do the testing, verify that things follow the steps and perform appropriate steps for implementations, or else could wind up with tail wagging the dog.
· Doesn’t disagree but sees working together as a useful way of constraining reference implementations.
· Useful to check behavior and mutually check on both sides.
· Agrees that specifications should drive implementations.
· But need to make sure we are still working with them so they know what they are being driven by.
· Who in this group is also on Ref Imp WG?
· John is on all call. We (GE) have basically provided the code for some of the transformations and our developers have been assigned to another task, so we are no longer doing reference implementation work, but if you’d like to have us more engaged to provide a check on things are being implemented, we could find a way to do that.
· We all could get involved in the writing of the specs (what these things should do) depending on how face to face goes.
· And by using the NHIN Direct wiki.
· Vince Lewis from his organization has been contributing. They touch base offline and Vince has been looking at the conformance guide. But Vince isn’t using it as a guideline.
· Might not hurt if on Tuesday morning at the Face to Face they check in with other workgroups during last ½ hour. Will make plans for more crossover.
· Wants to be on top of the issue Keith raised, the tail wagging the dog.
· Umesh and Sean wrote one of the implementations and first draft of specs, so there is a danger of not enough involvement from the documentation people.
· Need the process to be solid.
- àWill touch base with Brian Behlendorf.
Round: clarifying the difference b/w conformance guide and testing guide
||It appears both are important and something should be in place to have the conformance guide as a near-term deliverable, echoing Keith. |
||No comment. |
||Distinction between the two--what do we have in the conformance guide? Not sure if anyone has had a chance to look at it. Is that the level we should continue at? Not in terms of abstract model? Not clear on the language and the way we want to frame the guide. He understands testing guide is more about end-to-end interoperability works, but is still struggling with how to frame that conversation. |
· If I understand Keith correctly, conformance guide is defining on top of the specifications, conformance statements for what an implementation must do. For instance, language describing the “musts and shalls” for including an XDD file. It is used when there are specific components providing additional specifications about how to use.
· Testing guide provides set of end-to-end tests on much more of a script level. For example, if you start with internet message format with xyz components with abc trust circles, etc…you should expect to see X at the other end of the system. It is a set of test situations and test scripts that enable confirmation that a system is interoperable.
· That is what the conformance guide is talking about. John was challenging that. Is there an alternate framework we should look at for the conformance guide? There is a value 1 deliverable…so what are the next steps?
· Believes John was referring to a ref imp of security agent as opposed to more focus on conformance guide of specific interactions. John is changing what he has proposed and is in the process of making changes that will address Parag More’s question.
· Will wait for changes.
||Are we looking at different levels of conformance? Specification—so conformance guide defines different levels and provides testable statements. |
o If no, is a spec not enough? As in do I conform to specification or not?
o If yes, we would want to make sure anything in there is testable criteria.
· We are using unconditionally compliant and conditionally compliant = shall/shall not
· Same language used in all other circles
· Do we need separate specifications for conformance guide?
· APA, SMTP, client developer’s guide…doesn’t know what Doug means.
· What is going on the wire, which calls are made: how do I craft a conformance statement around what is here?
· Seems like Doug is describing the testing guide. If you do X action, Y will happen. Gives you the test set-ups (pre conditions), the test scenario and expected output.
· So performing such action shows test is passed?
· Where is a statement crafting the body of the message?
· Those are two different things: implementation guide shows how you craft it. Conformance guide says what the rules are. Rules don’t tell you how to do something. They are just the rules and have to be followed, without explanation. Implementation guide is really more about how/why to do something.
||Nothing to add. |
||Thought it was a lot clearer when conformance guide was defined as chunky rather than granular. |
· Thinks explanations make sense. If we want to keep testing guides as priority 2, we probably need to be doing that relatively late. Doesn’t know if the conformance guide is in a place where we could start the testing guide.
· Does anyone disagree? Anyone want to own the testing guide?
· Cannot declare the implementation done until we’ve got the conformance and testing guides done.
· We should focus on the conformance and testing guides during the –a-thons.
- Will provide additional assistance can't take responsibility for whole thing.
· Other stuff: priority two documents that don’t yet have owners.
· XDD developers guide moves down to priority three, leaving XDD implementation as priority two.
· Also, SMTP and script implementation guides.
· People in this group need to be working closely with implementation geographies.
· Email client considerations guide?
· Thinks fits with first category of guest imp/client imp.
· Thinks Will would be a good person to own and bridge the gap between Documentation and Testing WG and Implementation Geographies WG.
- What about John Moehrke?
· Three folks this morning at the Implementation Geographies WG meeting said they would go between the different a-thons, but cannot remember who.
· Still priority two, not priority one.
· Maybe putting out an outline or structure.
· Will write volunteer abstract, outline and start wiki page.
· Policy questions
· Janet Campbell will write higher level policy guide and will put out a call to people who want to get involved.
· Now have owners on the “2s”
· Can write abstract for FPD/XDD developer’s guide
· Question: we have two more people helping out from a staff perspective.
à Wants a list of docs we want pre-printed for view during document-a-thon.
· Going through the docs, annotating, looking at physical copies as a great process to getting to final documents you think are close to deliverable
· NHIN Direct Overview, Security, and Conformance Guide
- Content Security for Simple Heath Transport.
· Will be owner. Umesh and Sean wrote the first draft.
· Still call for as many edits done prior to the process as possible.
· Anyone want a document edited?
· So this means people feel the docs are fine, at the level they need to be at for Aug 17
·Encourage folks to go to the wiki page, start reviewing, offer comments.
· Use mailing list if you want people to review docs. There are people who want to help but can’t do full-time.
· Agenda for document-a-thon
o One document at a time
o Conformance guide
o Testing guide
o Communicating and representing to the ref imp group