Journal of Information, Law and Technology

Creating a Safety Net – A Proposed Rating Form
for Assessing the Quality of
Legal Information in Websites

Alan Robinson
Legit, Australia

legit@shoal.net.au

Delivered at the 2nd AustLII Conference on Computerisation of Law via the Internet, Australasian Legal Information Institute (AustLII), University of Technology, Sydney, Australia, 21-23 July 1999.

This is a conference paper published on: 29 February 2000

Citation: Robinson A, 'Creating a Safety Net – A Proposed Rating Form for Assessing the Quality of Legal Information in Websites', Conference Paper, 2000 (1) The Journal of Information, Law and Technology (JILT). <http://www.law.warwick.ac.uk/jilt/00-1/robinson.html>

Appendices
Appendix A - Ranking and Implementation of H.E.T.I. Criteria.

Importance (IMP) - reflects the importance of this criteria on assessing the quality of the information. 4 - Essential, 3 - Important, 2 - Desirable, 1 - Indifferent, 0 - Not at all. 
Ease of Implementation (Ease) - reflects how easy it will be to develop the criteria in a consumer useable way, 4 - very easy, 3 - moderately easy, 2 - minor difficulty, 1 - substantial difficulty, 0 - would not be able to do. 
Ideas on how to implement this criteria - how the criteria could be assessed by a consumer (e.g., under the criteria Credentials: Is the name of the author listed?(Yes/No) Are his/her credentials listed? (Yes/No) 				
	Ranking		How to Implement	
Criterion	IMP
(4-0)	Ease
(4-0)	How to implement this criteria	
Example
Credentials	4	3	 Is the name of the author listed? (Y/N)
·Are his/her credentials listed? (Y/N)
·How well do the credentials match the text? (5 – perfect match, 0 – not related) 	
C1 Credibility				
C1.1 Source				
C1.1a Source (e.g., organizational information)				
C1.1b Credentials				
C1.1.c Conflict of Interest				
C1.1d Bias				
C1.2 Context(e.g., advertising, medical condition)				
C1.3 Currency				
C1.4 Relevance/Utility				
C1.5 Editorial Review Process				
C2 Content				
C2.1 Accuracy				
C2.2 Hierarchy of Evidence				
C2.3 Original Source Stated				
C2.4 Disclaimer				
C2.5 Omissions Noted				
C3 Disclosure				
C3.1 Purpose of the site (e.g., promotional, educational)				
C3.2 Profiling (capture, sharing)				
C4 Links				
C4.1 Selection				
C4.2 Architecture				
C4.3 Content				
C4.4 Back Linkages and Descriptions				
C5 Design				
C5.1 Accessibility (e.g., can be used with low-end device)				
C5.2 Logical Organization				
C5.3 Internal Search Engine				
C6 Interactivity				
C6.1 Mechanism for Feedback				
C6.2 Chat Rooms (e.g., moderator present)				
C6.3 Tailoring (e.g., based on what algorithm, e.g., AHCPR)				
C7 Caveats				
C7.1 Alerts				

Appendix B - Pilot use of Legal Information Rating Form

Online Legal Information Rating Form - Lawstuff				
Criterion	How to implement this criteria 	Mark	Comment	
Accuracy	Does the information appear to be accurate?	1		
	Are there links to relevant legislation and case reports?	0		
	Is the original source stated?	0		
	Does the disclaimer describe limitations, purpose, scope, authority, and currency of information (0-5)?	4		
	Is it made clear that the information provided is not a substitute for professional advice?	1		
	Are any omissions noted?	0	6	
Source	How credible is the source? (see scale below)	4		
	Is the name of the author listed?	0		
	Are his/her credentials listed?	0		
	How well do the credentials match the text? (5 = perfect match, 0 = unrelated)	5		
	Is the content provided in the public's interest? 	1		
	Is any possible conflict of interest noted?	0		
	Does the information appear to be balanced?	1		
	Does the source appear to be unbiased?	1		
	Is the site not selling a product?	1		
	Is the site's purpose disclosed?	1		
	Is no user information captured (apart from feedback)?	1		
	Is privacy of personal information assured?	1		
	Is there an editorial review process?	0		
	Is the editorial review process explained?	0	16	
Currency	Is there a date stamp at the bottom of each page?	1		
	How current is the material? (5 = < one month, 0 = > two years old)	3	4	
Usability	How useful is the information? (0-5)	5		
	Are hyperlinks useful?	0 		
	Are hyper links properly identified, structured and authenticated? (0-3)	0		
	Is there a description of linked sites?	0		
	Is a graphical browser not required?	0	Overuse of graphics	
	Are plugins not required?	0		
	Which browser version is required (v2=3, v3=2, v4=1)?	2		
	Is the site logically organised?	1		
	Generic search engine = 2, Javascript engine = 1	0		
	Quality of search responses (0-2)	0		
	Feedback mechanism - email = 1, form = 2	2	Feedback by email & form	
	Is there a chat room?	0		
	If so, is a moderator present?	0		
	Is any information-tailoring algorithm disclosed?	0		
	Are users alerted when they move to an external site?	0	10	
			Total=36	

Online Legal Information Rating Form - F.C.A.				
Criterion	How to implement this criteria	Mark	Comment	
Accuracy	Does the information appear to be accurate? 	1		
	Are there links to relevant legislation and case reports?	2		
	Is the original source stated?	1		
	Does the disclaimer describe limitations, purpose, scope, authority, and currency of information (0-5)?	3		
	Is it made clear that the information provided is not a substitute for professional advice?	1		
	Are any omissions noted?	0	8	
Source	How credible is the source? (see scale below)	4		
	Is the name of the author listed?	0		
	Are his/her credentials listed?	0		
	How well do the credentials match the text? (5 = perfect match, 0 = unrelated)	5		
	Is the content provided in the public's interest?	1		
	Is any possible conflict of interest noted?	0		
	Does the information appear to be balanced?	1		
	Does the source appear to be unbiased?	1		
	Is the site not selling a product?	0		
	Is the site's purpose disclosed?	1		
	Is no user information captured (apart from feedback)?	1		
	Is privacy of personal information assured?	1		
	Is there an editorial review process?	0		
	Is the editorial review process explained?	0	15	
Currency	Is there a date stamp at the bottom of each page?	1		
	How current is the material? (5 = < one month, 0 = > two years old)	4	5	
Usability	How useful is the information? (0-5) 	5		
	Are hyperlinks useful?	1		
	Are hyper links properly identified, structured and authenticated? (0-3)	3		
	Is there a description of linked sites?	1		
	Is a graphical browser not required?	1		
	Are plugins not required?	0		
	Which browser version is required (v2=3, v3=2, v4=1)?	2		
	Is the site logically organised?	1		
	Generic search engine = 2, Javascript engine = 1	0		
	Quality of search responses (0-2)	0		
	Feedback mechanism - email = 1, form = 2	1		
	Is there a chat room?	0		
	If so, is a moderator present?	0		
	Is any information-tailoring algorithm disclosed?	0		
	Are users alerted when they move to an external site?	1	16	
			Total=44	

Online Legal Information Rating Form - L.A.C.				
Criterion	How to implement this criteria	Mark	Comment	
Accuracy	Does the information appear to be accurate?	1		
	Are there links to relevant legislation and case reports?	0		
	Is the original source stated?	0		
	Does the disclaimer describe limitations, purpose, scope, authority, and currency of information (0-5)?	2		
	Is it made clear that the information provided is not a substitute for professional advice?	1		
	Are any omissions noted?	0	4	
Source	How credible is the source? (see scale below)	4		
	Is the name of the author listed?	0		
	Are his/her credentials listed?	0		
	How well do the credentials match the text? (5 = perfect match, 0 = unrelated)	5		
	Is the content provided in the public's interest? 	1		
	Is any possible conflict of interest noted?	0		
	Does the information appear to be balanced?	1		
	Does the source appear to be unbiased?	1		
	Is the site not selling a product?	1		
	Is the site's purpose disclosed?	1		
	Is no user information captured (apart from feedback)?	1		
	Is privacy of personal information assured?	1		
	Is there an editorial review process?	0		
	Is the editorial review process explained?	0	16	
Currency	Is there a date stamp at the bottom of each page?	1		
	How current is the material? (5 = < one month, 0 = > two years old)	4	5	
Usability	How useful is the information? (0-5)	4		
	Are hyperlinks useful?	1		
	Are hyper links properly identified, structured and authenticated? (0-3)	2		
	Is there a description of linked sites?	1		
	Is a graphical browser not required?	1		
	Are plugins not required?	1		
	Which browser version is required (v2=3, v3=2, v4=1)?	3		
	Is the site logically organised?	1		
	Generic search engine = 2, Javascript engine = 1	2		
	Quality of search responses (0-2)	1		
	Feedback mechanism - email = 1, form = 2	1		
	Is there a chat room?	0		
	If so, is a moderator present?	0		
	Is any information-tailoring algorithm disclosed?	0		
	Are users alerted when they move to an external site?	1	19	
			Total=44	

Online Legal Information Rating Form - Law4U				
Criterion	How to implement this criteria	Mark	Comment	
Accuracy	Does the information appear to be accurate?	1		
	Are there links to relevant legislation and case reports?	1		
	Is the original source stated?	0		
	Does the disclaimer describe limitations, purpose, scope, authority, and currency of information (0-5)?	3		
	Is it made clear that the information provided is not a substitute for professional advice?	1		
	Are any omissions noted?	0	6	
Source	How credible is the source? (see scale below)	2	Two ex-legal aid solicitors	
	Is the name of the author listed?	0		
	Are his/her credentials listed?	0		
	How well do the credentials match the text? (5 = perfect match, 0 = unrelated)	2		
	Is the content provided in the public's interest?	1		
	Is any possible conflict of interest noted?	0		
	Does the information appear to be balanced?	1		
	Does the source appear to be unbiased?	1		
	Is the site not selling a product?	0	Selling links to private law firms	
	Is the site's purpose disclosed?	0		
	Is no user information captured (apart from feedback)?	0	Uses cookies	
	Is privacy of personal information assured?	0		
	Is there an editorial review process?	1	Outside lawyers used to check info	
	Is the editorial review process explained?	0	8	
Currency	Is there a date stamp at the bottom of each page?	1		
	How current is the material? (5 = < one month, 0 = > two years old)	4	5	
Usability	How useful is the information? (0-5)	5		
	Are hyperlinks useful?	1		
	Are hyper links properly identified, structured and authenticated? (0-3)	1		
	Is there a description of linked sites?	1		
	Is a graphical browser not required?	1		
	Are plugins not required?	0		
	Which browser version is required (v2=3, v3=2, v4=1)?	3		
	Is the site logically organised?	1		
	Generic search engine = 2, Javascript engine = 1	0		
	Quality of search responses (0-2)	0		
	Feedback mechanism - email = 1, form = 2	1		
	Is there a chat room?	0		
	If so, is a moderator present?	0		
	Is any information-tailoring algorithm disclosed?	0	14	
	Are users alerted when they move to an external site?	0	No alert given	
			Total=33	

Online Legal Information Rating Form - LawSoc				
Criterion	How to implement this criteria	Mark	Comment	
Accuracy	Does the information appear to be accurate?	1		
	Are there links to relevant legislation and case reports?	1		
	Is the original source stated?	1		
	Does the disclaimer describe limitations, purpose, scope, authority, and currency of information (0-5)?	5		
	Is it made clear that the information provided is not a substitute for professional advice?	1		
	Are any omissions noted?	0	9	
Source	How credible is the source? (see scale below)	4		
	Is the name of the author listed?	1		
	Are his/her credentials listed?	1		
	How well do the credentials match the text? (5 = perfect match, 0 = unrelated)	5		
	Is the content provided in the public's interest?	1		
	Is any possible conflict of interest noted?	0		
	Does the information appear to be balanced?	1		
	Does the source appear to be unbiased?	1		
	Is the site not selling a product?	0		
	Is the site's purpose disclosed?	1		
	Is no user information captured (apart from feedback)?	1		
	Is privacy of personal information assured?	1		
	Is there an editorial review process?	0		
	Is the editorial review process explained?	0	17	
Currency	Is there a date stamp at the bottom of each page?	1		
	How current is the material? (5 = < one month, 0 = > two years old)	4	5	
Usability 	How useful is the information? (0-5)	5		
	Are hyperlinks useful?	1		
	Are hyper links properly identified, structured and authenticated? (0-3)	2		
	Is there a description of linked sites?	1		
	Is a graphical browser not required?	0		
	Are plugins not required?	1		
	Which browser version is required (v2=3, v3=2, v4=1)?	2		
	Is the site logically organised?	1		
	Generic search engine = 2, Javascript engine = 1	1		
	Quality of search responses (0-2)	2		
	Feedback mechanism - email = 1, form = 2	1		
	Is there a chat room?	0		
	If so, is a moderator present?	0		
	Is any information-tailoring algorithm disclosed?	0		
	Are users alerted when they move to an external site?	1	18	
			Total=49	

Online Legal Information Rating Form - RLC				
Criterion	How to implement this criteria	Mark	Comment	
Accuracy	Does the information appear to be accurate?	1		
	Are there links to relevant legislation and case reports?	1		
	Is the original source stated?	0		
	Does the disclaimer describe limitations, purpose, scope, authority, and currency of information (0-5)?	4		
	Is it made clear that the information provided is not a substitute for professional advice?	1		
	Are any omissions noted?	0	7	
Source	How credible is the source? (see scale below)	4		
	Is the name of the author listed?	0		
	Are his/her credentials listed?	0		
	How well do the credentials match the text? (5 = perfect match, 0 = unrelated)	5		
	Is the content provided in the public's interest?	1		
	Is any possible conflict of interest noted?	0		
	Does the information appear to be balanced?	1		
	Does the source appear to be unbiased?	0		
	Is the site not selling a product?	1		
	Is the site's purpose disclosed?	1		
	Is no user information captured (apart from feedback)?	1		
	Is privacy of personal information assured?	1		
	Is there an editorial review process?	1		
	Is the editorial review process explained?	0	16	
Currency	Is there a date stamp at the bottom of each page?	1		
	How current is the material? (5 = < one month, 0 = > two years old)	5	6	
Usability 	How useful is the information? (0-5)	5		
	Are hyperlinks useful?	1		
	Are hyper links properly identified, structured and authenticated? (0-3)	2		
	Is there a description of linked sites?	1		
	Is a graphical browser not required?	1		
	Are plugins not required?	1		
	Which browser version is required (v2=3, v3=2, v4=1)?	3		
	Is the site logically organised?	1		
	Generic search engine = 2, Javascript engine = 1	0		
	Quality of search responses (0-2)	0		
	Feedback mechanism - email = 1, form = 2	1		
	Is there a chat room?	0		
	If so, is a moderator present?	0		
	Is any information-tailoring algorithm disclosed?	0		
	Are users alerted when they move to an external site?	1	17	
			Total=46