Computers in Psychological Evaluation

In the early 1980's a series of articles were published in education, medical and psychological journals that focused on the use of computers in education, health care and psychological evaluation. Computers were first used by psychologists to process data - usually true-false and multiple choice responses from paper and pencil questionnaires. This raw data was processed into meaningful scores and graphs that only trained professionals had access to and could interpret. As computers became more available, new software was developed that generated text that provided interpretative guidance to professionals. Computer based test interpretation (CBTI) systems for psychological evaluations were subsequently marketed to educators and health care professionals. Based primarily on industry issues, the American Psychological Association (APA) ethical codes were subsequently re-written to allow developers to protect proprietary intellectual property. External professional oversight of software performance was substantially eliminated in favor of internal business oversight. Professionals who used these programs no longer had a professional right to examine how a program worked or generated an interpretation of the results.

Studies on the use of computers in psychological evaluation and assessment recommended that without further advancement a computer could not reach diagnostic conclusions and fully replace face-to-face evaluation. Most state laws and licensing boards asserted and continue to assert than only licensed and qualified professionals can make a diagnosis. Despite these limitations, CBTI systems continued to evolve in support of providing information. But clinical results are still available only to licensed health care professionals. Clinical report software is sold only to licensed health care professionals who in most cases may not be trained at all in how to use these tools.

Clinical reports are currently not written for the lay person, but portions are shared with clients. Many professionals and organizations have taken a position that patients and parents may misunderstand CBTI and therefore could be harmed. Professionals are cautioned to not give patients reports. They are encouraged to go over the reports with patients in order to explain what the result mean. Such concern is certainly warranted for a CBTI system that is written for the audience of clinicians. It is important to recognize that the concerns over harm based on misunderstanding technical reports dissipate when the report is written specifically for adults (and not just clinicians) in the general population.

Computer based interviews, surveys, questionnaires, intakes and histories have been under development for years, but most of these simply process data, organize information, generate a document using patient record format and give professionals something to review. Software to perform these tasks has been developed to run on desktop computers in a professional's office or waiting room. These programs ask the patient questions about their history, attitudes and behavior. No commercially available programs have been offered that allow users to input and process information about their children and other people.

The developers of computer based interview systems initially experienced human resistance and practical obstacles - primarily because people were not comfortable with computers. More than one computer would also need to be available in offices and patient waiting areas. People are accustomed to completing a brief paper and pencil questionnaire on a clipboard. A computerized interview was complicated, unfamiliar to most patients and was expensive to develop. Answering a simple questionnaire on a computer in an office was far more difficult and complicated 10 years ago than it is today. The software back then had the unfortunate problem of not working all the time ("crashing"). Too many patients prove unwilling to come to an office appointment early just to use a computer in a waiting area. Software development proved more difficult than expected in order to design a computer based questionnaire that a person could complete quickly in one sitting prior to their appointment.

History of Internet and the Web

With access to the internet we can interact with other people, gather information and search for information in a knowledge-base almost instantaneously through fiber optics and a communication network of computers across the planet. The internet allows people to e-mail, voice and video-conference, transfer files and search "Web” for information." E-mail and the Web are available in most homes and even in the most remote areas of Alaska and the outback of Montana.

The Web has seen a great deal of change over the past decade. The technology has become faster and easier to use. Instead of taking 10 minutes, files and images can now be downloaded and displayed in seconds. Nearly a decade ago hundreds of companies and millions of dollars flocked to Web in order to develop "virtual" businesses that offered everything from clothing, to information, advice or to merely provide a presence on the Web were customers and stockholders could learn more about a company. Companies were formed to create new software and hardware products that allowed companies to conduct business on the Web.

Hiring a team of Web programmers and system engineers was necessary in the beginning. The cost to develop a simple Web site could be as much as $70,000. A simple Web site today can cost a few thousand dollars, and even less if you don't want to take a few hours and do it yourself. Web sites can be constructed as easily as using a word processing program. Years ago, sophisticated Web sites that interacted with data bases and processed credit card transactions initially cost anywhere from $250,000 to $500,000. There were many reasons why it cost so much. The primary problem has been that there are serious challenges that must to be overcome when trying to make software programs run over the Web. The Web was never designed to help process information but rather to search out and gather information that already existed in one standard format. Only a wealthy company, or wealthy speculators, could afford to develop and maintain a sophisticated commercial Web site.

The Need for Behavioral Health Resources on the Web

For the most part their problems go unrecognized. As a result they do not receive necessary services.

The formula for health care today does not address the unique or individual needs of teenagers. A diagnosis is almost always made based on impressions and a few very brief interviews. Even when children have mental health insurance coverage, the "gatekeepers" who authorize payment for services tend to deny or limit services to a significant degree. Few children will ever receive necessary and comprehensive evaluations. These services are not covered by their health insurance. Most children are minimally screened and the first option is increasingly to medicate children for the psychological consequences of overcrowded school, abuse, neglect, family dysfunction, poverty, divorce, grief, alcohol and drug use. Very little time is spent evaluating the real cause of their emotional and behavioral problems.

Many of the children of parent who can actually afford medication will stop primarily because they do not see the benefits. More importantly, the side effects such as insomnia, weight gain, impotence, headaches and nausea are not worth it in their mind. Even more alarming has been recent discoveries that the side effects of starting and stopping medications can be dangerous. For more information see,

• Antidepressant Use in Children, Adolescents, and Adults - FDA warning

Finding the right help is a daunting task for parents who know there is a problem but can't describe what they see in a manner that will be taken seriously. Parents are often forced to appeal to health care providers who are prone to either minimize parental concerns or they side with children who typically minimize their behavior and omit important information. Many parents are not trained to advocate effectively to care-givers and insurance companies that profit by minimizing the amount of care provided.

What kind of help do children with behavioral and emotional problems need? Parent education, family counseling, a psychological evaluation, a treatment program, boarding school, medical care or psychotherapy? This list is nearly endless. Parents must choose from and communicate with numerous professionals including counselors, clinical social workers, psychologists, family nurse practitioners, psychiatric nurse practitioners, neurologists, family physicians, psychiatrists and adolescent psychiatrists to find help for their child’s problems. Parents become lost without the information and guidance necessary to find help for their child. Effective documentation and description of their child's history and behavior is a critical element to help parents work effectively with health care professionals. Parents in remote isolated areas do not have access to mental health services. Poverty represents a tremendous obstacle for parents who cannot afford an evaluation. Parents with children "at-risk" frequently can't wait for a thorough face-to-face screening or clinical report. There are not enough professionals in this country who are willing or able to work with children who have behavioral and emotional problems.

Use of the Internet in Mental Health Screening and Educational Consulting

Finding safe, effective and ethical sources of information about treatment and treatment programs for youth at-risk can be a challenge. Many parents and professionals are going to the internet and Web to find support and solutions for troubled teens. Web sites are being used by parents and professionals searching for educational, intervention, emotional growth and treatment resources for youth. Likewise these programs are making their presence known on the Web.

How Parents Currently use the Web

The Web provides an excellent opportunity for parents and professionals to compare, evaluate, and "shop" for professional services, information resources and programs. Parents can go directly to a Web site by entering the Web site address directly (also call a URL) or they may use a search engine to locate information on a variety of topics. The primary uses of the Web by parents and professionals include,

• Researching problems. This might include gathering information on parenting, problem behavior, specific disorders and treatment approaches. 

• Finding professionals and programs.  Parents use the Web to locate professionals locally and nationally. Parents also use the Web to locate boarding schools and treatment resources such as residential treatment programs, therapeutic boarding schools, traditional boarding schools, evaluation centers, wilderness therapy programs, outdoor schools and adventure based programs. 

• Investigating a specific professional or program. Many programs have Web sites that provide information regarding their programs and services. Few professionals maintain Web sites that provide detailed information about their background and practice.

• Finding consumer protection information. This might include finding News Room, articles and other information resources that are intended to support public health, safety, referrals and the delivery of professional services.

How Programs Currently use the Web

Programs use the Web for a variety of purposes. The Programs use the Web primarily to generate referrals and to encourage parents and professionals to contact programs. Secondary purposes are to facilitate admissions, public relations and to foster better communication between programs, professionals and parents.

Differences between programs can be tremendous. In addition, the amount of information and detail provided by programs on their Web site can vary a great deal. This can make it difficult to investigate programs on the Web. Contact information and a general overview is usually provided on most Web sites. A few programs provide detailed information pertaining to the program's ownership, structure, staffing, philosophy, cost, policies and procedures. The best Web sites provide complete, detailed, clear and honest descriptions of their program.

The Web is also being used by schools and treatment programs to gather information from parents that can be used to evaluate the appropriateness of a child's admission. A few Web sites are using questionnaires that "screen" prospective youth and offer "advice" regarding the appropriateness of admitting a child. Close examination will reveal that there are two types of screening and advice systems available on the Web. The first are those that almost invariably recommend placement in the program that pays for the Web site. The second types are those that don't recommend programs but instead offer a brief interpretation and recommend the user obtain qualified help.

Many of these so called screening Web sites seem to be no more than a ploy developed by marketing people who are completely unqualified and remain anonymous despite inquiry. Many of these questionnaires are manifestly unfit for their proposed use. Parents reaching one of these Web sites are lured into completing a brief questionnaire after which the results are submitted on-line. An automated reply is returned with feedback and recommendations. Screening questionnaires that are available on the Web are very simple, provide little or no data processing, and reflect only the simplest form of results. The real difference between these questionnaires is whether or not the results are a ploy intended to scare parents into admitting their child into a program that funds the Web site.

The Future of Educational Consulting and Mental Health Screening on the Web

In order to envision the future of software programs that operate on the Web we must first consider the programming languages and “platforms” available to us. There are many methods, but some lend themselves to the Web more than others. Brief screening tools that offer little information and recommend the Web site’s treatment program should become a thing of the past. More powerful and ethical approaches are possible.

Computers can already be programmed to process histories and behavioral rating information quickly. Data processing is easy. Generating individualized reports and recommendations is not. The type of computer system necessary to educate, guide and empower parents is referred to as "expert systems." A search of the top search engines would suggest that there does not appear to be any expert systems for health care operating on the Web.

An expert system is a software program that can be designed to simulate the logic of people such as teachers, physicians, psychologists or psychiatrists. There are a number of ways to accomplish this. A programming language designed specifically for could be used, or a language could be used based on a data processing language that can implement Bayesian, Boolean and "fuzzy" systems.  There are two additional and essential elements that are more important than the language used. One must hire a knowledge engineer and then put this person to work with an expert in a domain of knowledge (a psychologist, physician, etc...). Knowledge engineers are essential because they have the skills to define decision processes and to consider the relevant information used by professionals.

The process of developing can be lengthy and expensive. Developers must be persistent and well funded. A new advisory or expert system in a domain of knowledge will invariably fail, but subsequent revisions and refinement can increasingly and accurately approximate conservative professional judgment. Finding an expert and knowledge engineer who can work effectively together is the biggest challenge. Finding an expert who is also a knowledge engineer is nearly impossible.

Expert systems, as defined here, are sophisticated screening applications for mental health that can be used as follows.

• To identify individuals who exhibit behaviors associated with psychiatric disorders.

• To identity patterns of behaviors associated with specific behavioral, emotional and psychological problems that are not psychiatric disorders.

• To identify individuals who exhibit specific behaviors associated with a risk of violence, injury, accidental death or suicide.

• To monitor and evaluate progress or deterioration in behavior.

• To provide users with education and recommendations necessary to seek qualified consultation and evaluation.

• To educate parents, caretakers and professionals regarding referral, intervention and treatment options.

• To help users document a child’s history, behavior, problems and needs.

• To support parents, caretakers and professionals who are seeking information and resources for youth with emotional, behavioral and psychological problems.

• To provide guidance and options for professionals conducting interventions for youth at-risk.

Questionnaires developed by a psychologist working with a knowledge engineer can provide an objective approach to evaluation by directing the rater’s response in a reliable manner to specific questions supported by meaningful scales that are associated with clinically valid content. Systematic interviews on-line can minimize inferences and biases that create errors in omission or the overemphasis of behavior and historical information that can create errors in conclusion. The raters or "informants" completing these questionnaires must know the child, be capable of reading the questionnaire and they must answer all mandatory questions. In most cases only a few questions can be answered "Don't Know" for the results to be valid. Failure to correctly answer questions provides a less reliable and therefore less valid and useful report.

The use of behavioral ratings and checklists is already a commonly used method for assessing children, adolescents and adults. These approaches can effectively identify symptoms and signs of educational, behavioral and mental health problems. Behavioral observations and ratings over time and from multiple sources can provide information about the progress of treatment and whether or not a child’s condition or behavior is getting worse.

Any expert system focused on behavioral health will necessarily utilize empirical and clinical evaluation procedures. This will mean that existing scales to measure behavior must be incorporated and new scales must be developed. Standardized measures as well as other valid clinical methods must be blended into the expert system's procedures. This is not a task for an inexperienced knowledge engineer, programmer or expert.

Ensuring privacy and security are important factors when developing an expert system for the Web. The ability to provide security as well as threats to security has grown over the past decade. The internet and Web was not initially designed for security. Privacy and security were after thoughts. But unlike a year ago, data gathered on the Web can now be protected by powerful encryptions that are virtually unbreakable. The chances of someone intercepting and reading a transmission is nearly zero because encrypted data could take a hundred years to decipher. Actual Web sites can also be certified "Hacker-Safe" daily (99.9% safe) by external computer security services that can run over 2000 internal security checks daily. The next generation of operating systems (the software than runs other software) is already being designed specifically to eliminate hacking. It may soon be far easier to break into a building and steal a computer from a bank vault in broad daylight.

Summary

More than simply gathering and processing information, the Web can be used to educate and empower people to make decisions and to seek further help in a timely manner. The real advantage of developing online systems is that of saving time and the ability to gather a great deal of information quickly and process that information is a reliable manner with valid results. Professionally written reports based on extensive face-to-face interviews and surveys can take a professional three to four weeks to complete. Time is always a factor when parents are dealing with a youth at-risk. A properly designed expert computer system will promise reliable, valid and useful results that can be generated and displayed on a parent’s computer screen confidentially and in seconds. This process could empower parents and give them information based on everything they know. Parents are no longer completely dependent on professionals who may not be available. And while online screening services cannot diagnose children, or replace face-to-face evaluations, the result can be used by qualified consultants, educators and health care professionals to guide and refine subsequent placement, evaluation and treatment decisions. This can save time. With online expert systems, consulting and evaluations that would otherwise take a month, could take place in a day or less than a week. The use of a Web based expert system, in the comfort of a parent's home, can provide parents with a reliable means to document their concerns and to support subsequent educational and health care decisions. Best of all, reports and recommendations that normally cost over $1200 can be generated for 5 to 10 percent of that cost.

Dated: June 12, 2003