The Use of an Acceptance Test for Accessing an Electronic Information System

Introduction

In Norway, a national electronic information system for critical information sharing (CIS) between levels of care was implemented in the period 2013–2017. All general practitioners (doctors), doctors in emergency units in primary care, and doctors in emergency rooms (ER) in specialist services were required to perform an e-learning course and pass an acceptance test in order to access the CIS. CIS is nationally rooted and determined. Its purpose is to give fast and safe access to information about health. We have studied the preparation for the e-learning course and the acceptance test, the use of the system, and the project leaders’ implementation work.

Study focus

The acceptance test was devised as a door opener to the use of CIS. Passing the test gives doctors a personal identity card (PKI- Public key infrastructure) to access the system. For our research, 12 project leaders were interviewed about their preparations for acquiring the doctors’ license, and 25 doctors were interviewed about their use of CIS. The research question for this part of the study was: What were the informants’ experiences with the acceptance test for accessing use?

User experiences

Problems with Java proved to be an obstacle in taking the test. Moreover, after passing the test, the PKI card had to be picked up in person at a specified office.

Some of the project leaders’ experiences with preparing the doctors accessing the test are captured here:

I met some challenges, non-updated Java, and…when arriving at the acceptance test…you had to put the identity card in the keyboard and finish the e-learning course, and you enter the URL for the acceptance test, but it all stops because of an out-of-date Java version. Several people are quite critical of the acceptance test. The hardest with it is actually to come so far seeing the content in the test. It’s not the test itself. It’s banal. The technical problems have been challenging. Picking up the card…It takes too much time to start to use it. #9

If I had been able to decide, I would not have required the test. To get people using the system faster … E-learning would have been enough preparation [for CIS]. #6

Some of the doctors’ experiences are captured here:

E-learning… I feel resigned to it…we need to spend time on it… It does not always feels rational. They know how hard it is for us to get time for training and courses… I feel controlled from the top: instructed to take these e-learning courses…. This course is banal. It is basic stuff about confidentiality and data security. #4

We were ordered to take the course. #8

It was Java-based… I tried to upgrade Java, but I could not make it work. It is stupid because people could not access it in the beginning. #17

I cannot look it up every time because it takes too long to log in… It’s not that easy. It takes about 30 seconds. That does not sound so time consuming, but in relation to consultations it is. #23

The acceptance test was easy in itself, but the users met technological challenges and found that it was time consuming.

Discussion and Conclusion

The acceptance test was seen as problematic as the users were challenged by the technology and the adjustments to the local information system. It was also too time consuming for the practitioners from the perspective of their other work as it involved taking the test, picking up the PKI card, and logging on for use in critical situations. They experienced the medical content in the test as easy, but the tool itself was technologically complicated and time consuming. The use of CIS requires passing the test, which is determined by the national authority in Norway. The professionals feel controlled and ordered to take the test. As its content is similar to the principles they follow in daily practice without using CIS, they believe that the acceptance test should not be needed in order to use the information system.