Messaging within OpenMRS


Contents

Food for thought

Justin - Paul and I were discussing messaging within the system and came up with the following four distinct types of messages within the system. Whether these are handled within one API or four is up for discussion. And perhaps you see the world differently...just wanted to provide our thoughts as you worked out the specs. -Burke 16:01, 19 January 2006 (US Eastern Standard Time)

  1. System messages — e.g., "Server is going down in 10 minutes"
    • We imagine these would be stored in a business table and retrieved through a Context.<em>getSystemMessage()</em> function and rendered at the top of every web page.
  2. Alerts — e.g., "Your patient XXXX-X had a critical result (Cr 4.5)"
    • These are critical messages that require confirmation of receipt. We imagine these would be stored in a business table with a status attribute (e.g., read/unread). The API would expose these as User.<em>getAlerts()</em> and/or User.<em>getAlertCount()</em>. Users would have to see these as soon as they logged into the system. If alerts are not read in a timely manner, additional alerts/events might fire. If we got fancy, we might allow for multiple-choice responses to alerts and/or links for specific actions — but we would not want to support/emulate any kind of e-mail functionality (i.e., you can confirm you saw the alert, maybe pick a particular action, but not respond).
  3. Messages"Your patient XXXX-X was seen in RADIOLOGY CLINIC today"
    • These are convenience "informative" messages directed at users/groups and would be directed to an external e-mail server (ideally maintained within the same intranet). This is where a MessagingService API could be created with methods like MessagingService.<em>send(Collection<User> who, String subject, String message, int priority)</em>. A MessagingService.<em>getMessageCount()</em> method would allow web applications to display a convenient "You've got mail" teaser.
  4. Events"Patient XXXX-X registered into database"
    • Internal system messages within/between modules/applications. We imagined the EventService API would be used to generate and receive HL7 event mesages.

Darius's Comments

  • I feel like internal system messages are a completely seperate beast, and don't have the same data model or behavior
  • I'd been thinking we should maintain messages for users internally to the OpenMRS system, the main motivation being that we can guarantee security. Were you thinking of storing those on a seperate email server application, but providing access from within the OpenMRS web app? Or were you thinking about saying "You've got mail" but when the user clicks on it, we launch their normal email client?
  • An alternate way to implement the Alerts would be using scheduling/event queue, but this might be overly sophisticated.
    • Deliver a regular message
    • Schedule a check-whether-read or check-whether-actioned check for X time in the future
    • etc.
    • (I suppose the problem is that this doesn't work with an external mail server.)

So to sum up, what I'd originally envisioned was that all messages (if delievered in-system) would be like what you describe as Alerts. I think I like your breakdown better, but I'll ruminate.

Burke's Responses

I think we're criss-crossing our terminology here. Let me blabber incessantly to attempt clarification. :o)

  • What do you mean by "internal system messages"? Are you referring to System Messages or Events? In any case, we see both of these as modeled completely differently than messages/alerts to users.
    • System messages would come for a simple table (date/time, creator, expire date/time, message) and would show up on every web page.
    • Events would handle all of the messaging between applications. Again, a completely different model from user messages/alerts. Events are the system events/triggers that listeners are listening for and responding to. All the HL7 stuff like ADT (admit, discharge, transfer) messages, new obs created, patient registered into system, etc. Basically, events are computers talking to computers.
  • We agree that Alerts should be handled within OpenMRS — i.e., messages that require confirmation. But we really don't want to try and build an entire e-mail application. There are plenty of open-source and commercial e-mail packages that can do a much better job. So...
    • Alerts represent the mission-critical messages to users that must be confirmed. We envision limiting functionality for alerts to:
      1. Display a list of alerts for the user
      2. User confirms that you he/she saw the alert
      3. And, just maybe (if we got fancy), we'd allow for a link in the message or some way for the user to say "Yes or No" (not a version 1.0 feature)
    • Messages would be basically e-mail notifications. Convenient, but delivery does not need to be guaranteed. We would rather have an API to send these off to another application. We envision Squirrel Mail or some other e-mail package being set up alongside OpenMRS and an OpenMRS configuration file being tweaked to point user messages to the e-mail server. OpenMRS would have the ability (through the API) to POP the message count from the e-mail server to display a teaser. But the teaser would be a link that opens a new window containing the e-mail app. Many of the user messages (e.g., "your patient was seen by Dr. So-in-so" or "You have 15 unsigned orders") do not really need confirmation and would be better served up in a familiar e-mail fashion, reserving Alerts for the truly high priority messages.
  • The Event stuff is what you guys were originally talking about for messaging — i.e., the internal system events/triggers to be shuffled between objects and their listeners. We don't really care if you want to call these "Messages" and rename messages to users as "UserMessages" ... just want to be clear about what we're discussing.
  • Event messaging would be tied up with transactions and deep within the bowels of the API. Alerts and Messages would be two APIs for delivering messages to users (confirmation-required and non-confirmed, respectively) — the latter using existing e-mail technology. Finally, System Messages would serve up system-wide announcements from administrators. So, Events might trigger Alerts or Messages. Reading an Alert after you login, could potentially generate a UserReadAlert Event message. Clear as mud?
    • Perhaps better terminology might help...
      • System Messages --> Administrator Announcements
      • Alerts --> User Alerts
      • Messages --> E-mail (too medium-bound) or User Messages
      • Events --> Messages (I think this is what you originally were thinking of as "Messages")

Finally, I think your "Messages" are our "Events." When Paul was saying that we could use messaging functionality sooner than later, he was talking about user notification through Alerts/Messages (as described above) rather than the listener-notifying internal system messaging that you guys have been contemplating, trying to merge with transacations, etc.

I don't think that (using our terminology) Events and Messages/Alerts should go through the same plumbing. We may end up with an org.openmrs.event.* package with classes for various Events. Messages just needs a service with send-to-smtp & get-message-count-from-mail-server functions. Alerts would need an API and tables to support display and confirmation-of-receipt of text-based notifications to users of the system.

-Burke 17:53, 19 January 2006 (US Eastern Standard Time)

Message/Alert Data Model (first pass)

We need to support something like:

  • send(Message, Collection<Recipient>)
  • getUnreadMessageCount(User) // if user is using an external email server for messaging, then check that
  • getUnreadMessages(User) // only get messages from internal messaging system
  • getUnactionedAlertCount(User) // logic for this may be complicated: find any alerts for any patient that this user is watching
  • getUnactionedAlerts(User)

Different ways of delivering messages:

  • In-system messages
    • Messages that live in the OpenMRS system, and whose read/unread is tracked there.
    • All Alerts are delivered this way, possibly with email notification as well
      • Read/Unread is not interesting for alerts. Actioned/Unactioned is.
  • Trusted Email
    • Administrator should be able to designate some email servers as trusted, meaning that end-to-end encryption is guaranteed between messages from the system to the user. (e.g. a server on the intranet that allows webmail access over SSL only)
  • Untrusted Email

message_template

Holds information about the template used to dynamically generate messages. The template text field will probably contain variable substitution parameters that will need to be evaluated by the template engine.

 CREATE TABLE message_template (
   message_template_id int(11) NOT NULL auto_increment,
   template longtext NOT NULL
 );


message

 CREATE TABLE message (
   message_id int(11) NOT NULL auto_increment,
   message_template_id int(11) NOT NULL,
   created_date datetime NOT NULL default '00000000 00:00:00',
   patient_id int(11),                 Patient that the message pertains to
   encounter_id,
   obs_id
 );

message_sent

Keeps track of the message that is being sent, but only saves information about the message being sent (not about each recipient).

sender_id might be a user, event, etc (can be null) sender_type_id references a table that does not exist yet action_status refers to the action taken (viewed, skipped, etc)

 CREATE TABLE message_sent (
   message_sent_id int(11) NOT NULL auto_increment,
   message_id int(11) NOT NULL,
   body longtext NOT NULL,
   subject text NOT NULL,
   message_sent_type_id int(11) NOT NULL,
   sender_id int(11),                         
   sender_type_id int(11),                    
   action_status <datatype>?                  
 );

Why don't we just combine message and message_sent into one table. What's the point in separating them? Djazayeri 01:03, 23 January 2006 (US Eastern Standard Time)

message_sent_type

 CREATE TABLE message_sent_type (
   message_sent_type_id int(11) NOT NULL auto_increment,
   name text NOT NULL,
   description text NOT NULL
 );


message_sent_recipient

Actual message recipients. After the list of recipients is pulled via the distribution list tables, each message is sent and receipt is stored in this table. This allows us to keep track of who has read or taken action on a particular message/alert.

 CREATE TABLE message_sent_recipient (
   distribution_list_entry_id int(11) NOT NULL auto_increment,
   message_sent_id int(11) NOT NULL,
   user_id int(11) NOT NULL,
   date_read datetime NOT NULL default '00000000 00:00:00',
   date_sent datetime NOT NULL default '00000000 00:00:00',
   sent_flag tinyint NOT NULL default 0,
   internal_flag tinyint NOT NULL default 0 -- i.e. is this message sent using the system's own messaging system, as opposed to external email.
 );

message_sent_list

 CREATE TABLE message_sent_list (
   message_sent_list_id int(11) NOT NULL auto_increment,
   message_sent_id int(11),
   distribution_list_id int(11) 
 );


distribution_list

 CREATE TABLE distribution_list (
   distribution_list_id int(11) NOT NULL auto_increment,
   name varchar(100) NOT NULL
 );


distribution_list_entry

 CREATE TABLE distribution_list_entry (
   distribution_list_entry_id int(11) NOT NULL auto_increment,
   distribution_list_id int(11) NOT NULL,
   user_id int(11),
   user_group_id int(11),
   relationship_type_id int(11)
 );

--Jmiranda 14:10, 21 January 2006 (US Eastern Standard Time) Edited: Djazayeri 01:03, 23 January 2006 (US Eastern Standard Time)

More of Burke's blabber

Justin,

I think we need some more discussion on what our goals are for messaging. I understand that here we're modeling just "Messages" and "Alerts" (i.e., system-wide announcements and software event propagation will be handled separately).

Among messages/alerts, I see only two basic types of messages:

  1. Messages that must be confirmed
    • This includes simply "confirm message read" or "force selection of an action" — i.e., "read" can be seen and treated as an action, right?
  2. Message that need not be confirmed
    • I think we would just as soon send these out to an SMTP server of the administrator's choosing. I suppose you would like to include these so we are not dependent on an outside application, right? If so, could we design the internal "mail" system as an independent module that could serve as a poor man's e-mail delivery system? That way you could choose between internal vs. external mail apps? Alerts (messages needing confirmation) would always be delivered internally (whether or not a duplicate message was generated to be sent via mail).

Would it be sufficient to determine whether a message needed confirmation by message type? And how about if we use user groups instead of creating distribution lists?

message       /* a record for each message sent that we want to track (all alerts +/- messages) */
  message_id
  message_type
  from
  to          /* string for easy display */
  subject
  message     /* completed message (i.e., template with values filled) */

user_message  /* used to map alerts +/- messages to recipients */
  message_id
  user_id
  status      /* read, unread, action taken, etc. */

message_type
  message_type_id
  name
  description
  priority    /* alert vs. informational message */
  actions     /* for alerts: some string that can easily be parsed to determine actions choices to offer */
  template    /* template for generating message */  

With an API something like:

/* for sending out messages (both alerts and unconfirmed messages) */
void sendMessage(User[] to, String subject, MessageType messageType, Object[] params)

/* CRUD operations */
create/get/update/deleteMessageType(...)

/* to get alerts (messages needing confirmation) and take action(s) */
Message[] getUnconfirmedAlerts(User who)
void confirmAlert(User who, Message message, String action)

/* for interacting with mail system -- whether internal or external */
int getUnreadMessageCount(User who)
String getMessageSystemUrl(User who)

I'm pushing to separate "Alerts" (messages that must be confirmed or acted upon) vs. "Messages" that are for informational purposes and confirmation is not needed.

  • Alerts would be displayed every time the user logged into the system.
  • Messages would be rendered as a teaser ("you've got mail") and clicking on that link would take you to the "mail" system, which might be our internal mail display module or a full-fledged e-mail client set up by the administrator. While you might want to create your own e-mail module, I would argue that a simple module with an inbox for display and delete functionality would do the job; anybody wanting more functionality could point the API to an external e-mail client instead.

-Burke 13:21, 23 January 2006 (US Eastern Standard Time)


Ben's Late Input

Burke, Paul and I talked today about needing to get feedback to our data assistants. This could be easily done by creating an Alert (as defined in Burke's last blabber). This is in-system, no outside email needed. The only user response allowed for an alert is "ok".

An alert would not be too complicated to create:

 CREATE TABLE `alert` (
  `alert_id` int(11) NOT NULL auto_increment,
  `user_id` int(11) default NULL,
  `role` varchar(50) default NULL,
  `text` varchar(512) NOT NULL,
  `date_to_expire` datetime NOT NULL default '0000-00-00 00:00:00',
  `creator` int(11) NOT NULL,
  `date_created` datetime NOT NULL default '0000-00-00 00:00:00',
  `changed_by` int(11) default NULL,
  `date_changed` datetime default NULL,
  
  PRIMARY KEY (`alert_id`),
  
  KEY `alert_creator` (`creator`),
  KEY `alert_assigned_to_user` (`user_id`),
  KEY `alert_assigned_to_role` (`role`),
  KEY `user_who_changed_alert` (`changed_by`),
  
  CONSTRAINT `alert_creator` FOREIGN KEY (`creator`) REFERENCES `users` (`user_id`),
  CONSTRAINT `alert_assigned_to_user` FOREIGN KEY (`user_id`) REFERENCES `users` (`user_id`),
  CONSTRAINT `alert_assigned_to_role` FOREIGN KEY (`role`) REFERENCES `role` (`role`),
  CONSTRAINT `user_who_changed_alert` FOREIGN KEY (`changed_by`) REFERENCES `users` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT=;
  CREATE TABLE `alert_read` (
  `alert_id` int(11) NOT NULL,
  `user_id` int(11) NOT NULL,
  `date_read` datetime NOT NULL default '0000-00-00 00:00:00',
  
  PRIMARY KEY (`alert_id`, `user_id`),
  
  CONSTRAINT `alert_read` FOREIGN KEY (`alert_id`) REFERENCES `alert` (`alert_id`),
  CONSTRAINT `alert_read_by_user` FOREIGN KEY (`user_id`) REFERENCES `users` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT=;

An alert can be sent to one user with `alert.user_id`, or to all users with a certain role `alert.role`. I realize you may want to sent to an arbitrary group of users, but tough luck, that will come in version 2 (or send a message).

When you log in all non-voided/non-expired/non-read alerts that you match will display on the top of every page (in an overflow div, so as not to take up the entire page if there are a lot of them). In order for the alerts to disappear you must click the box next to them -- essentially saying you read the alert. (Claiming you read the alert populates the `alert_read` table with that `alert_id` and your `user_id`.)

Justin's response

I like your idea about the overflow div. We were discussing something similar today (04/12/2006) with regards to our lab data entry system. We think that it will be important to notify doctors, data entry clerks, etc immediately when a condition in the system causes an alert to be raised. This includes the following actions:

  1. Send a message to the user's browser (via ajax) as you suggested
  2. Send a message to phone(s), pager(s), or email(s) (via the message service) based on the user's preferences.

Obviously, the phone/pager/email message will contain non-specific information (i.e. "You have an alert about a patient with a low CD4 count"). The doctor will need to log into the system in order to view specific information about the alert.

Here's a use case:

  1. Lab technician enters a lab result into the system
  2. System stores lab result
  3. System runs MLMs over lab result
  4. System raises alert
  5. System saves alert
  6. System sends message to appropriate parties

--Jmiranda 21:46, 12 April 2006 (EDT)

Darius's response

The reason I wanted to allow messages to be sent to groups of users is because I thought we had User-Group-Role in the data model, as we talked about at AMIA, rather than just User-Role. I see that's not in the data model yet, so we have no problem waiting on that.

Djazayeri 23:17, 12 April 2006 (EDT)


Ben's reponse

Theres where you're in luck. Roles are grouped into roles, so can still send a note to, say, all Providers easily. What you can't do is send something to Data Assistant #3, and Clinician #2 and #6.

-Ben 23:21, 12 April 2006 (EDT)