Publication Internet integrated in the daily medical practice within an Electronic Patient

within an Electronic Patient Record

Christian Lovis a,b, Robert Baud b, Jean-Raoul Scherrer b

a

Department of Internal Medicineb

Division of Medical Informatics

University Hospital of Geneva, Switzerland

email lovis@dim.hcuge.ch

Publication

Internet integrated in the daily medical practice within an Electronic Patient Record.Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

Address of author:Christian Lovis

Department of Internal Medicine and Division of Medical InformaticsUniversity Hospital of Geneva21, rue Micheli-du-CrestCH - 1211 Geneva 14Tel +41 22 372 61 40Fax +41 22 372 62 55

Email lovis@dim.hcuge.ch

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

page 1/10

Abstract

Healthcare enters the information age and professionals are finding an ever-growing role for computers in thedaily practice of medicine. However, a number of problematic issues are associated with electronic publications,especially through Internet. Whilst access to any information has been improved, access to specific informationhas become more and more difficult [1], due to the lack of a general meta-knowledge allowing to structure Inter-net resources. Physicians have to learn and adapt themselves to computers and Internet, but Internet has to meetthe specific requirements of Healthcare. Important issues must therefore be addressed to allow a real and dailyuse of Internet in the medical practice.

The paper discusses most of these issues and proposes a solution developed at the University Hospital of Genevathat integrates an Electronic Patient Record with Internet, without compromises on security or on performancesand that runs on standard PCs’.

Keywords: Internet, Integration, Healthcare, Electronic Patient Record

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

page 2/10

Introduction

Introduction

The 1997’s edition of the American Medical Infor-matics Association Annual Fall Symposium (for-merly SCAMC) has been dedicated to Internetunder the title of \"Is healthcare ready for the Inter-net ?\" The maieutic*, Socratic way to answer thiscould be \"Is the Internet ready for Healthcare ?\" IfInternet has to become a major tool in the physi-cian’s daily work, and it will for sure, it has to bemore adapted to medical domain. Healthcare re-quires confident tools working round-the-clock thatare easy to use, fast, reliable and allow to get the

form on-line searches (that does not implies thatthey are able to get usable results !).

Many questions rise when speaking about Internetand Healthcare. Among them, one can emphasiselimited transmission speed, undependable knowl-edge longevity, poorly documented post-publicationediting, copyright infringement, poor validation andcultural impediments to acceptance.

Problems and considerations

The use and the transfer of data through Internetgenerate many problems. Some of them address thecopyright impacts, as \"downloading\" a document or

Medical knowledgeNot mandatoryPresently never ensured on InternetPresently never ensured on InternetUnclear on InternetSometimes definedSecurityRelevanceValidationResponsibilityAuthorshipPatient dataMandatory. Possible in Intranet.Ensured by the legal responsibility ofhealthcare providerEnsured by the legal responsibility ofhealthcare providerClear : the authorAlways definedTable 2: Properties of medical dataright response without hurdle. The ease of use ismade of two very different but complementaryelements: firstly the interface has to be user-friendlyand the layout should have some common charac-teristics and secondly the informational contentshould be well structured, with a meaningful or-ganisation to the medical domain. This is a majoran image is essentially making a copy of thatdocument or image. In that case, copyright laws andthe rights that they afford authors are involved [3].Other questions involve the responsibility ofauthors or the performance of query engines. Whatabout the potential consequences due to bad orinsufficient information ?

ProtocolTransfer perform-anceEncryptionAuthenticationPrivacyActivity auditData structureUser-InterfaceControlled resourcesOften proprietary, needs middleware to beused or widespreadMostly LAN or WAN with excellent per-formancesPossiblePossibleEnsuredPossibleControlledControlledWWW resourcesStandard hypertext mark-up language(HTML), with the limitation of the variousversions.General net, mostly using phone lines, telecomlines, unpredictable performancesSubmitted to state law & authorisationDifficultAbsentPossibleAnarchyUnpredictableTable 1: Differences between an HIS WAN/LAN and Internetissue, especially when taking into account that theskills of junior staff for information technologiesare overall overestimated [2]. Murphy et al. showedthat only 6% of junior staff is able to use the emailand that less than 50% is technically able to per-*

Greek : maieutikos, from maieuesthai, to act as midwife, from

Information on Internet can be divided in twogroups. The first one refers to private data, inHealthcare these are patient data, whereas the sec-ond group refers to general or specific informationin medicine, from textbooks knowledge to journalarticles or guidelines. These two groups of infor-mation have very different characteristics. Some ofthe key properties that distinguish patient data and

page 3/10

maia, midwife, nurse

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

Problems and considerations

general knowledge that can be found on Internet aresummarised in the table 1 and key technical differ-ences are collected in table 2.Security

The security covers many different aspects that canbe divided into two groups; the first one applies tothe actions that can be taken on the data whereasthe second group is related to the data themselves.Security about actions on data

The level to which programs, data or devices aresafe from unauthorised use is particularly importantfor private patient data. It involves the privacy (thestate of being free from unsanctioned intrusion) andthe confidentiality. Confidentiality needs two stepsto be ensured, the identification of both the re-quester and the server and their authentication. Thislast point is closely related to electronic signature,that should identify the author and the readers ofany document, as well as the integrity of its content.Another important function is the availability ofmonitoring. Such monitoring must ensure that allaccesses and actions can be tracked. In our experi-ence, all accesses to patient private data are gath-ered in a document that belongs to the medicalrecord itself and can be viewed by all authorisedusers, including the patient. The prevention of ac-cess of non-authorised persons to identifiable in-formation includes also encryption of data for theirtransmission.

Security about Data

Healthcare information requires integrity, that isprevention of modification, deletion and loss ofinformation; the receiver being sure that the data hehas in hands is complete and authentic. Additionalimportant points are accessibility and availability.The assurance that data is available in a short time,at any time at the good place. These elements areespecially important when thinking about recoveryprocedures and redundant hardware to ensure un-interrupted services. All these issues should, how-ever, never hamper the reliability of data byhealthcare providers that may need them in emer-gency situations, as well as its access by the patientitself [4, 5].

There is also a very important perspective in beingable to ensure an automatic anonymous extractionof data. The ability to furnish huge clinical data-bases for researches purposes, case-based reasoningor reimbursement claim databases will becomemore and more important. One must be able toensure that the distributed data no more contain anyinformation that could identify the concerned pa-tients. Ensuring anonymous extraction of data is anobvious task when dealing with structured data, but

it remains an unresolved problem when workingwith text documents [6].Relevance

Patient data must be pertinent where facts are in-volved; whereas linked external medical knowledgemust be relevant to the current patient. Relevancerelies on pertinence to the matter at hand. It in-cludes the ability of an information retrieval systemto select and retrieve data appropriate to user’sneeds, that includes the conformance to currentmedical knowledge and the appropriateness to thequery done. Retrieved knowledge must be up-to-date. Many web servers supply an outdated infor-mation. In healthcare, that could cause inappropri-ate patient handling with unforeseeable conse-quences. Knowledge must therefore be docu-mented, i.e. date of publication, releases, authors,sources, references, etc. Knowledge should even, inmost cases, be criticised and evaluated. Such workis being done presently, for example within theEvidence Based Medicine and Cochrane Initiative[7]. There is a clear need for a meta-knowledge ableto describe, index and structure the medical knowl-edge in Internet. One should be able to evaluate thequality of queries, in term of a) specificity thatmeans that adequacy between the query and theresults, but also in term of b) sensitivity that meansassessing which is the proportion of all relevantdocuments that have been retrieved. Initiatives likethe Unified Medical Language System (UMLS)Metathesaurus, especially in their recent works onnatural language indexing, could play a major rolein that situation, similarly to the one played forMedical Literature [8]. To get highly specific re-sults requires a strong model of the semantic. Sucha model is currently being developed within theGALEN European project [9]. Furthermore, theGALEN project includes natural language analysisand natural language generation in a multilingualapproach that is essential in the context of sharableElectronic Patient Record (EPR) [10, 11, 12]. Thereare currently no available solution that addressesthese issues. Meanwhile, we have built an Intranetserver that has actually approximately 9’000 HTMLpages. These pages have been produced and aremaintained under the responsibility of each Clinicsof the General Internal Medicine Department. Foreach involved clinic, persons in charge have beenchosen and are responsible for producing the inter-nal Web documentation as well as maintaining linksto World Wide Web links in their domains. In ad-dition to this manual work, we use a search enginethat has been specifically developed for the medicaldomain. The Health On the Net Foundation and theMolecular Imaging and Bioinformatics Laboratoryat Geneva University Hospital have developedMARVIN, a robot that searches sites and docu-ments specifically related to a given specialised

page 4/10

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

Problems and considerations

field. One such robot has already been implementedfor health and medicine. A medical dictionary of12,000 words has been built. Each word has beengiven a weight describing its relevance and speci-ficity in documents for health and medicine. Theweights have been obtained by performing a statis-tical evaluation of all words contained in a prese-lected set of 1,000 medical documents. MARVINsearches the Web and selects only documents thatare relevant to the health and medical fields. Docu-ment relevance is computed according to a formulathat takes into consideration the number of wordsfrom the medical dictionary that are found in thedocument, as well as their associated weights. Se-lected documents are stored into a database thatmay be queried through MedHunt, the Health Onthe Net Foundation’s medical search engine [13,14].

Validation

Internet contains more and more information ofinterest for patients and healthcare professionals.Identifying the outstanding resources from thechaos is difficult [15]. This point is largely dis-cussed actually in medicine, especially within thefield of Evidence-Based Medicine. Strong method-ology has been published to evaluate medical publi-cations in peer-reviewed magazines, as can be seenin the Users’ guide to the medical literature in theJournal of American Medical Association (JAMA)series [16, 17]. Whereas a lot has been writtenabout validation in the field of knowledge-basedsystems or decisions supports systems in medicalinformatics [18], few can be found about the as-sessment and the quality control of Internet re-sources. Therefore, it is unlikely that electronicinformation published on Internet will replace sci-entific publishers soon and it explains why elec-tronic journals seem to have limited influencewithin scientific discourse [Erreur! Signet nondéfini.]. Uncritical acceptance of information ap-pearing in the electronic media is prevailing andelectronic publishing may occur without the rigor-ous peer-review process common for publishing inscientific journals [19]. The problem of quality ofdata and ease of query is well summarised by aprospective study done by Bigsby with students ofhealth information science. Results show: a) Evenunder pressure to use Internet, libraries remain thereference source of choice for research; b) Internetprovides entertainment appeal even though practicalutility remains limited; c) Technological profi-ciency with HTML and search engines is perceivedas an asset; d) Varying availability of Internet re-sources, uncertain and varying quality of sourcesand limited specificity of research results are themajor disadvantages of Internet. Therefore, mosthealthcare people don’t trust in that information[20]. Some Web servers now address those issues

with success and provide the rapid collection anddissemination of quality, peer-reviewed pertinentmedical information [21], either specialised in spe-cific subdomains like resource for anatomopathol-ogy [22] or emergency medicine [23] amongstother. Those intending to publish material elec-tronically must accept responsibility for the veracityof the information, realising that anyone, from thelay consumer to the professional, may have accessto that information, unless closed users groups areset (Intranets).

Human-interaction issuesStructure

One of the hindrances most frequently raised whenteaching the use of Internet to physician staff is thelack of formal organisation and presentation stan-dards. More and more, one assist to a kind of \"mul-timedia\" race, with sounds, animated pictures, Javaapplets downloads that bewilder the users and con-tribute to increase loading time for the Web pagedespite the amelioration of the \"information super-highway\". Within our private Intranet, we followstrict rules for the knowledge organisation as wellas for the layout using HTML templates. The prob-lem arises when accessing to Internet pages. Theparsing of HTML pages in order to extract the trueknowledge within the HTML layout has not beenrealised. In order to try to keep a coherent context,all links in our Intranet that accesses Internet sitesare embedded within a frame structure that preventsusers from being lost.User-interfaces

An easy-to-use interface is defined as \"a user-friendly system designed for small businesses withno prior computer experience\" The use of Internetresources becomes daily more difficult. It is notonly because interfaces per se are difficult to use,but also because more and more prior knowledge isrequired to get it out. The problem raises both fromthe browser side and the content side. Nowadays,the popular Internet browsers are more and morecomplicated to use (see version 4 of MicrosoftInternet Explorer or the by Netscape Corporation).Menus configure themselves or just may disappear,alternate windows appear or, for example, dialogboxes give constantly advertisement on securityissues, sometimes not very understandable for neo-phyte users (\"The server tries to download a cookie.Do you want to accept it? If not, the page may notbe displayed\" no comment…). But this is only theeasiest aspect of the problem. We currently teachthe use of Internet in our Clinic. The main problemencountered is due to the total absence of standardsin the graphical interface between various Internetplaces. Most people soon feel bewildered and loose

page 5/10

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

DOMED, the Integrated Electronic Patient Record

a lot of time trying to catch some logic behind allthose interfaces. After some attempts and a lot oftime spent, most prefer to abandon the use of Inter-net in their daily work, just use it to play duringcoffee-break or use only very few bookmarks onknown places. For these persons, it is necessary to\"pre-process\" resources available through Internet.This can be achieved either using powerful queryengines, like the MedHunt engine available de-scribed above, or by offering a structured web sitethat proposes an organised and critiqued list ofpertinent links to medical knowledge.Integration

It is important for clinical practice to be able toretrieve relevant literature and to keep informed ofthe state of medical science. Ideally, this should beclosely linked to the daily work, and especially tothe problems encountered with patients, and notonly done as part of continuing medical education.A way to partly accomplish such integration is tomerge the system used to access the patient datawith an Internet Browser. Many projects are cur-rently developed that use Internet and HTML toaccess both the Electronic Patient Records (EPR)and Internet resources. There are clearly two op-tions. One option is to use Internet browser tech-nology to access patient data, and the other is tointegrate browser functionality in an existing EPR.Due to unresolved issues discussed earlier, wedecided to integrate an Internet browser in the EPR,therefore trying to bridge the gap between patientdata and medical knowledge but keeping a closecontrol on the Internet aspect.

As issues of security, accountability and access areresolved, the Internet will act as the central infra-structure for a global as well as a community healthinformation network.

in our Institution. It allows easy access to a varietyof clinical data from patient records, includingimages from the hospital-based picture archivingand communication system, laboratory results,procedures and diagnoses encoding and free text,like admission/discharge letters or daily notes.Integration of WWW navigational tools with auto-matic links to Internet documents and other elec-tronic resources is driven with a knowledge-coupling mechanism. The knowledge-couplingmechanism currently used is quit simple. It is basedon morphosemantem analysis of input sentence togenerate normalised words that will serve as querysentence for the search engine. This mechanismallows users to select any part of the EPR, likesentences or words in a discharge letters, and have aquery automatically sent on Internet or other re-sources (Medline, online textbooks, etc.), the resultsbeing displayed within the EPR and associated withthe current patient. These links are not stored in the

InternetSecuredcommunicationsExternal providersHospital Information SystemStandardcommunicationsCD-ROM, IntranetTechnical & semantical middlewarePatient dataMedicalKnowledgeDOMED, the Integrated Electronic Pa-tient Record

Electronic Patient Records are becoming telemedi-cal and multimedia documents managers. Theyshould accompany the patients their whole lifetimeand be able to collect data from many differentorigins, to show them in various forms and toautomatically sort these data in a medically perti-nent way. Special requirements are arising to fulfilthese demands. Flexible data models with explicitcontents based on open communication structuresare the cornerstones of this approach. It is com-monly described as technical and semantic mid-dleware. A supplementary step consist of providingthe integration of patient data with Internet re-sources within a unique common user-interface (asin figure 1).

A software package called DOMED is currentlydeveloped for the display and manipulation of EPR

KnowledgecouplerProprietary EPRwith InternetbrowserFig. 1: Internet and Patient data integration

current implementation. The integration has beenbuilt on a double structure. The first point consist ofquerying our Intranet. Unsuccessful queries will

page 6/10

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

DOMED, the Integrated Electronic Patient Record

generate reports that may lead to add some knowl-edge to the Intranet. The next point is to propagatethe query on Internet. This query is carried outusing the MARVIN multi-agent retrieval systemthat is shortly described above.

The Internet technology and associated develop-ments, like the Java Programming Language hasprovided new tools for the development of EPR[24]. A major preliminary work has been to choosebetween two main options, taking into account thatall points discussed earlier: a) Develop an EPRusing Internet technology, and integrating into this,using Java and Common Gateway Interface, allfeatures needed that are not provided through stan-dard Internet browsers. b) Building a full applica-tion with all needed features embedded and inte-grating a full, Java compliant, Internet browser into

This way to do allows to optionally use completelydifferent protocols to handle patient data or unpro-tected knowledge resources, whilst keeping aunique interface for the end-users.Key issuesPerformance issues

The EPR needs local computation, including imagemanipulation [25], natural language processing,filtering, querying, data structuring, scientific chartswith trends computing and decision support. Thisprocessing can only be performed with acceptableresponse-time in native processor language. Thecurrent Java interpreters as well as Just in TimeCompilers have not reached that level of perform-ance. In our experience, Java performs too slow,

Daily follow note

Medical

guidelines onInternet

Medical history always visible

Figure 2: Integration of patient Data and Internet resources

it. As showed on figure 1, the second solution hasbeen chosen. This approach has some importantadvantages, as there is no infringement on securityor authentication of healthcare data, flexibility ofrecord software, ability to integrate many tools, likenatural language ICD encoder, numerical data ana-lysers, and so on.

especially when working with real-time applicationand transactional databases, even if it is well suitedto the needs of EPR and can interface with clinicaldata repositories written in SQL. The developmentitself has shown to be quit easy, as we benefit frommodern development tools and rich object classes.Technical issues

The HIS of the University Hospital of Geneva isbuilt on relational databases in an open and distrib-

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

page 7/10

Conclusion

uted UNIX architecture [26]. In this architecture,there is a unique common identification system forusers and patient data (administrative, laboratory,reports, PACS) is available through unique andcommon services. These services are currentlybased on the TCP/IP protocol and remote processescontrol to execute SQL queries on the databases.Commercial object-oriented middleware layers, likeCORBA, are currently being evaluated. An Thistechnical middleware allows easy integration ofmany resources providers. A lot of work as still tobe done to associate external providers like generalpractitioners or other hospitals. But several politic,ethic and administrative hindrance have first to beget over. On this strong layer, around fifty localdatabases running on small PC’s and Macintosh arecurrently being normalised and centralised. TheEPR application itself is therefore strictly inde-pendent from the data sources. Any new providercan be included in the technical middlewarethrough proprietary protocols if necessary, thendescribed in the semantic middleware, to be thenintegrated in the EPR. It does not impose any com-munication data transfer protocol on other providersthat may not have the technical resource’s to mi-grate on new environments. Despite this, all trans-action through the www remain possible and com-pletely integrated. The Internet is to be consideredas a data source amongst all other in the possibledata sources of the EPR.Security issues

All points discussed earlier concerning securitymay be applied in such a development. One of themost important points is that healthcare authoritieswant to be sure that patient data could benefit fromhigh security levels. This point is extremely impor-tant for user acceptance, as data protection remainsa very emotional point for most people, especiallyconcerning healthcare. An interesting point of hav-ing integrated the browser in the EPR is to have fullcontrol on it. So, we can restrict the access to Inter-net to those web sites that are of concern for medi-cal field and that have a reviewed content whilstletting a free access to the whole Internet only inlimited cases.User interface

The integration of all EPR tools as well as Internetbrowser in a unique system allows to have a veryuniform and constant user-interface, therefore sim-plifying greatly the teaching work and the accep-tance. Because the links between patient data andInternet resources are made transparent and auto-matic, it is very natural to browse indifferentlypatient data or Internet data (figure 2). In that case,a very important problem that remains is that in-formation browsed in the Internet about a givenpatient at some time may no more be reachable

some time later. As this could have important ethi-cal and medico-legal consequences, we are actuallydiscussing about having the possibility to providean automatic mechanism that copies the content ofexternal resources into the patient data. This allowsa better a posteriori comprehension of what ele-ments influenced the decision making.

Conclusion

The use of Internet in the daily practice encountersmany problems; the most important of them beingthe amount of time spent on searching valuable andpeer-reviewed information. Another essential pointinvolves the security, in its large sense, security ofdata, including privacy, integrity, reliability andsecurity of identification of users, includingauthentication. A good way to encompass theseproblems is to build an Internet browser within theEPR, instead using Internet to browse the EPR.This allows to have two main streams for data, onebeing reserved for patient data, secured, whilst theother one is for Internet accesses. These two differ-ent kinds of accesses are transparent for the users,as both are done within the EPR. Accesses to Inter-net are driven through a knowledge coupler thatpermits queries directly from texts, like dischargeletters or follow notes. Other accesses to Internetcan be done through a well-organised site thatstructures links to validated medical sources scat-tered in the world. Use of Internet outside thosevalidated sources is time-limited. The integration ofthe EPR and Internet can be done without compro-mises for security within the same user-friendlyinterface. The prototype presented here has beenpresented to the clinicians of our Institution andreceived an enthusiastic approbation. It will bewidely used in the daily medical practice nextautumn. A project within the Geneva State to ex-tend this development to general practitioners iscurrently in a design and decision phase.Acknowledgment

MARVIN has been supported in part by the SwissNational Fund for Scientific Research under grant #21-43501.95. The Galen consortium is supported bythe European Union, Directorate XIII, Brussels, andfor the author’s group by Office Fédéral del’Education et de la Science, Bern, Switzerland.The DOMED project is supported by the GenevaState PL-7512.

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

page 8/10

Conclusion

S

ummary

Healthcare enters the information age and professionals are finding an ever-growing role for computers in the daily practice of medicine. Computers areused for research, education, medical record keeping, communications, reference

resource and decision support amongst other. However, a number of problematic issuesare associated with electronic publications, especially through Internet. Whilst access toany information has been improved, access to specific information has become more andmore difficult, due to the lack of a general meta-knowledge allowing to structure Internetresources. With this incoming technology, one has to think about whom has to adapt towhom. Physicians have to learn and adapt themselves to computers and Internet, butInternet has to meet the very specific requirements of Healthcare. Important issues musttherefore be addressed to allow a real and daily use of Internet in the medical practice.The paper discusses most of these issues and proposes a solution developed at the Uni-versity Hospital of Geneva that integrates an Electronic Patient Record with Internet,without compromises on security or on performances and that runs on standard PCs’

The use of Internet in the daily practice encounters many problems; the most important ofthem being the amount of time spent on searching valuable and peer-reviewed informa-tion. Another essential point involves the security, in its large sense, security of data, in-cluding privacy, integrity, reliability and security of identification of users, includingauthentication. A good way to encompass these problems is to build an Internet browserwithin the EPR, instead using Internet to browse the EPR. This allows to have two mainstreams for data, one being reserved for patient data, secured, whilst the other one is forInternet accesses. These two different kinds of accesses are transparent for the users, asboth are done within the EPR. Accesses to Internet are driven through a knowledge cou-pler that permits queries directly from texts, like discharge letters or follow notes. Otheraccesses to Internet can be done through a well organised site that structures links to vali-dated medical sources scattered in the world. Use of Internet outside those validatedsources is time-limited. The integration of the EPR and Internet can be done withoutcompromises for security within the same user-friendly interface. The prototype pre-sented here has been presented to the clinicians of our Institution and received an enthu-siastic approbation. It will be widely used in the daily medical practice next autumn in theGeneva University Hospital. A project to extend this development to general practitionersis currently in a design and decision phase.

References

1 23 4 5 6 7 8 9 10 11 12 13

S.D. McLeod, J. P. Gieser. Knowledge or noise. Scientific publication and the electronic journal. Arch Ophthalmol 114(10):1269-1270 (1996)

J. Murphy, R. Errington, S. Achara, R. Simon, M. Olaiwon. Preparing the next generation of Clinician to manage information. InProceedings of Medical Informatics Europe 1997, technology and Informatics (43), C. Pappas et al. Editors, IOS Press, 676-680(1997)

E. J. Stern, L. Westenberg. Copyright law and academic radiology: rights of authors and copyright owners and reproduction ofinformation. AJR Am J Roentgenol 1(5):1083-10888 (1995)

F. H. Roger France. Control and use of Health Information : a doctor’s perspective. Int. J. Biomed 43: 19-25 (1996)

R. Jones, S. McGhee, A. Hedley, K. Murray. Patient access to information. Current Perspectives in health Computing, 206-212(1988)

H. Bouzelat, C. Quantin, L. Dusserre. Extraction and anonymity protocol of medical file. Proc AMIA Annu Fall Symp 323-327(1996)

http://www.hcn.net.au/cochrane /ebm.htm

B. L. Humphreys, W.T. Hole, A.T. McCray, J.M. Fitzmaurice. Planned NLM/AHCPR large-scale vocabulary test: using UMLStechnology to determine the extent to which controlled vocabularies cover terminology needed for healthcare and public health. JAm Med Inform Assoc 3(4):281-287 (1996)

A.L. Rector, S. Bechhofer, C.A. Goble, I. Horrocks, W.A. Nowlan, W.D. Solomon. The GRAIL concept modelling language formedical terminology. Artif Intell Med 9(2):139-171 (1997)

J.C. Wagner, J.E. Rogers, R.H. Baud, J.R. Scherrer. Natural Language Generation of Surgical Procedures. To be publishedMedinfo 1998

M. Carlsson, H. Ahlfeldt, A. Thurin, O. Wigertz. Terminology support for development of sharable knowledge modules. Med Inf21(3):207-214 (1996)

A.M. Rassinoux, J.C. Wagner, C. Lovis, R.H. Baud, A.L. Rector, J.R. Scherrer. Analysis of medical texts based on a soundmedical model. Proc Annu Symp Comput Appl Med Care 27-31 (1995)

The MARVIN Multi-Agent Vagabond, http://www.hon.ch/MedHunt/Marvin.html

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

page 9/10

Conclusion

14 15 16 17 18 19 20 21 22 23 24 25 26

Health on the Net Foundation, http://www.hon.ch/HomePage/Home-Page.html

W. M. Silberg, G. D. Lundberg, R. A. Musacchio. Assessing, controlling, and assuring the quality of medical information on theInternet: Caveant lector et viewor--Let the reader and viewer beware [editorial]. JAMA 277(15):1244-1245 (1997)

G. H. Guyatt, D. L. Sackett, D. J. Cook, for the Evidence-Based Medicine Working Group. Users’ guide to medical literature, II :How to use an article about therapy and prevention, A : are the results valid ? JAMA 270: 2598-2601 (1993)

C. D. Naylor, G. H. Guyatt, for the Evidence-Based Medicine Working Group. Users’ guide to medical literature, X : How to usean article reporting variations in the outcomes of Health services JAMA 275: 5-558 (1996)

U. G. Gupta. Validation and verification of knowledge-based systems: a Survey. J Applied Intelligence 3:343-363 (1993)R. Sikorski, R. Peters. Internet anatomy 101. Accessing information on the World Wide Web. JAMA 277(2):171-2 (1997)

D. J. Bigsby, J. R. Moehr. Internet for teaching and learning introductory health informatics. Proc Annu Symp Comput Appl MedCare 532-536 (1995)

I. Benjamin, J. W. Goldwein, S. C. Rubin, W. G. McKenna. OncoLink: a cancer information resource for gynecologic oncologistsand the public on the Internet. Gynecol Oncol 60(1):8-15 (1996)

J. C. Fournet, V. Colin, P. Bertheau, J. F. Emile, F. Jaubert, J. P. Jais, N. Brousse. Internet, une nouvelle ressource pour l’anatomiepathologique? Ann Pathol 16(6):478-483 (1996) (http://www.anapath.necker.fr)

L. G. Yamamoto, P. J. Suh. Accessing and using the Internet’s World Wide Web for emergency physicians. Am J Emerg Med14(3):302-308 (1996)

R. P. Rodgers. Java and its future in biomedical computing. J Am Med Inform Assoc 3(5):303-7 (1996)

O. Ratib, Y. Ligier, C. Mascarini, M. Logean, C. Girard, G. Trayser, D. Hochstrasser. Multimedia image and data navigationworkstation. Radiographics 17(2):515-521, 522-524 (1997) (http://expasy2.hcuge.ch)

J.R. Scherrer, C. Lovis, F. Borst. DIOGENE 2, a distributed hospital information system with an emphasis on its medical infor-mation content. IMIA Yearbook 1995 of Medical Informatics, The Computer-Based Patient Record, International Medical Infor-matics Association (IMIA), J. H. van Bemmel, A. T. McCray Editors, Schattauer Verlagsgesellschaft, Germany, 86-97, 1995

Internet integrated in the daily medical practice within an Electronic Patient Record. Lovis C, Baud RH, Scherrer JR.Comput Biol 28:567-579, 1998

page 10/10