Visual Programming and Software Engineering Group

How to use the WWW-based tool to create "smart documents"

Disclaimer: this tool is at an early prototype stage and is mostly meant to be a proof of concept. Your comments are welcome.

The tool we are demonstrating is meant to be used by WWW document authors who want to include, in their documents, input fields that the document reader (the user) can fill in, and that will lead to different other WWW documents based on the (free text) input that the user has given. This tool is heavily based on forms. The author as well as the reader of such "smart documents" will therefore need a form-capable WWW browser. In spite of this, the authors do not need to know how to put forms in their documents, as our tool will do it automatically for them.

The author can specify criteria that will be matched against the user input to classify this input in different categories. All the possible input that match a given category will trigger the same behavior, i.e. will lead to the same document. There is no limit as to the number of different categories.

The syntax of the criteria is rather simple and allows the author to specify patterns to match against the user answer. The mechanism is based on the detection of (possibly multiple) keywords within a sentence, with the possibility to provide lists of synonyms and to specify only partial words, and even to specify words that should not be found in the sentence.

Here are the various steps involved in the creation and use of such smart documents. It does not involve running a special HTTP server, nor downloading a special program. Everything can be done via WWW, with a regular browser supporting forms.

The author first prepares the document that will contain the user input field, putting a placeholder at the place where the input field will later be automatically substituted. This document should be accessible on some HTTP server to which the author has access. An example of such a document is available, in which the placeholder is the word AnswerForm. This placeholder could be just any sequence of characters.
The author prepares the file containing the various criteria and the association list relating each criterion with the URL of a document that should be displayed when the criterion matches the user's input.
The format of the file is the following:
- Each criterion starts with a line like <<CriterionName>> and ends with a line like <<END CriterionName>>. Between these two delimiting lines, there can be any number of lines defining the rules to match against the user's answer.
  Example:
```
<<No>>
_NO_|_STOP_|_ENOUGH_|_EXIT_|_OUT_|_FINISH_|_END|_NEGATIVE_|and|not|_YES_
_OF COURSE_|_SURELY_|and|_NOT_
<<END No>>
```
- Similarly, an association list starts with a line like <<PlaceHolder>> (the same symbol as the one used in the HTML document that will contain the user input form), and ends with a line like <<END PlaceHolder>>. Between these two delimiting lines, there can be any number of lines defining a "CriterionName-URL" pair. These lines must start with CriterionName, which must be the name of one of the criteria defined in the file. The criterion name is followed by a space character and a URL. Note: if the criterion name is missing (the line starts with a space followed by a URL), it is assumed to be a "catch all" criterion, that is, a criterion that always succeeds. This is meant to be the last line of the list, to catch all possible unexpected answers.
  Example:
```
<<AnswerForm>>
Thermometer http://cuisung.unige.ch/FreeText/thermom.html
Instrument http://cuisung.unige.ch/FreeText/instrument.html
Hand http://cuisung.unige.ch/FreeText/UseHand.html
 http://cuisung.unige.ch/FreeText/Unknown.html
<<END AnswerForm>>
```
The author invokes the form in http://cui.unige.ch/eao/www/FreeText/DeveloperForm2, and fills in the details about his/her document:
- the URL of the original document in which a placeholder should be substituted,
- the URL of the file containing the various criteria with their associated web documents,
- the placeholder that the author chose to put in the original document,
- the size of the input field that should be substituted to the placeholder.
There is an example form that has been pre-filled to show how the form should be filled. We suggest that you open another browser window on this example form to try it while you keep on reading the present document (with NCSA Mosaic, use the middle mouse button to follow a link to a document that will appear in a new window. With Netscape 1, you will have to select the File/New_Window menu item, then use the Go/Info_on_how_to...smart_documents to see the present document in a second window, then follow the link to the example form).
When the author presses the Get me the final document button in the form, the content is sent to a cgi-bin script named developer2.
The developer2 cgi-bin script first checks that the document containing the placeholder can be retrieved over the net, then checks that the placeholder can be found in it. At each of these steps, an appropriate error message is sent back if a problem occurs.
The developer2 cgi-bin script then replaces the first occurence of the placeholder in the original document with the HTML code for a form that contains a text-input field, a "Continue" button, and hidden fields containing the criteria found in the criteria file (also retrieved accross the net). For the moment, no check is made to ensure that the syntax of the criteria is correct (this is a prototype).
Try clicking on the Get me the final document button in the second window and see what you get.
The resulting document is then sent to the author as a reply to the query.
The author can try this resulting document right away to see if it behaves as desired. If not, the author can update the criteria file and go back to the form (using the Back button of the WWW browser) to get a new version of the resulting document.
Try entering something like "I use my hand" or "I use an instrument", or even "I guess a thermometer would do the trick" and see what reply you get (don't forget to type the Return key or to click on the "Continue" button, and watch the document title of the reply, among other things). Go back to the previous document and try to enter another answer. Watch how you get each time a different reply (assuming that you type a different answer each time).
When the author is satisfied with the result, he/she can save the document (in HTML form, of course) in a file that is accessible from his/her HTTP server to make it available to other users.
When a user reads this generated document and fills in the input field, the content is sent to another cgi-bin script, named Analysis.
(You saw what happened when you tried typing in various answers)
The Analysis cgi-bin script invokes our pattern matching program which will tell which criterion was satisfied. If the pattern matching works properly, the cgi-bin script will issue a "Location: ..." reply with the URL of the document corresponding to the category of the user input. This will cause the WWW browser to fetch that document automatically. If, for some reason, the pattern matching program chokes on one of the criteria provided by the author, the cgi-bin script will warn the user, asking him/her to tell the author about the problem.

Syntax of the criteria

The syntax is described in a separate document.

CGI-bin scripts used in this demo

The following scripts are written in Tcl.

Developer's form handling used by a teacher to insert a user input field in a document, with embedded answer analysis criteria.
Answer analysis, invoked when the user fills in the user input field in the teacher's final document. It calls the Pascal program that does the actual pattern matching
http-query-setup.tcl is a Tcl script used by both scripts above.

You can install the developer's form handling script on your own WWW server, if you want to have better response time. But you are welcome to use our server too. As for the Answer analysis script, it wouldn't help to install it on your server if you don't install at the same time the Pascal program that does the pattern matching (see below).

Pattern-matching Pascal program

The pattern matching is done by a program written in Pascal, that works as a filter: it gets from "standard input" a sentence followed by an arbitrary number of patterns, and returns in "standard output" the number of the pattern that matched (1 if first pattern matched, ..., n if n-th pattern matched, 0 if no match was found). You can either use the on-line version that we put on the web, and that can be used from anywhere on the Internet even if the documents that use it are on other servers, or, if you belong to an academic institution and want it for non-profit use, you can get in touch with us to get the sources (~300 lines of code).

Our next step will probably be to translate the pattern matching Pascal program into a Java applet that could run directly in a (Java-enabled) browser.

Related work

As far as we can tell, previous work focusing on providing educators with tools to embed answer analysis in WWW documents has been limited to multiple choice questions, true/false questions, single numeric questions, and single algebraic expression questions. We have found at least the following:

The Tutorial Gateway, by Neal M. Holtz, Civil Engineering Department, University of Carleton.
Mklesson, by David A. Wheeler, Institute for Defense Analyses (IDA) in the Computer and Software Engineering Division (CSED).

Additional reading

The following papers describe some aspects of the system:

Free-Text Answer Analysis on the Web (compressed version, uncompressed version),
invited paper for the Revue Informatique et Statistique dans les Sciences Humaines (RISSH) journal.
Advanced Educational Uses of the World-Wide Web (compressed version - uncompressed version)
presented at WWW'95, in Darmstadt, published in Computer Networks and ISDN Systems, Vol. 27, No. 6, Apr. 1995, ISSN 0169-7552, pp 871-877. (Postscript version, 38Kb).

If you would like to comment on or give us some suggestions about the information we're providing, you can do so by following this link

Bertrand Ibrahim -

Last modification: April 12, 2000.

Site Hosting: Bronco