wiki:UserWorkContext

TracNav?

User Work Context

Detecting the current workflow of the user by observing the operations the user does. Operations are notified to the User Observation Hub, listeners to this hub can detect the current work context of the user.

It also stores the observe NOP (Native Operations) into the "context" storage in RdfRepository, at the moment when they are received.

Installing and Running it

Users:

Developers:

To train the BestPerformingPredictor, upload Claudia's historical user context:

Authors

  • Julien Gaugaz
  • Sven Schwarz
  • Leo Sauermann

Documentation

The service can be divided in three parts that work together:

  • Observing user operations the user via user observation plugins that send messages to the UserWorkContext service.
  • Gathering user operations in the UserWorkContext service and computing and updating the current mid-term context thread
  • Using context in the GUI and to support the users

Javadoc

Observing user operations and context

Applications and services should observe what the user is doing and other operations and data that influence the working context in which the user is situated in the moment. The observations are expressed using the NOP Ontology for User Observations and Native User Operations:

We have several plugins that observe the actions of the User and create signals in the NOP ontology, these plugins capture events like clicking a link, opening a file, etc. More is documented at the specialized project (hosted in cooperation with the MyMory project)

Native operations can be sent to the UOH or directly to the UserWorkContext service. See the usercontext wiki for directions.

Current Work Context

Current work context. Based on the observed user operations, the current work context is computed. The work context is represented in the UserContextOntology. The ontology models objects and attributes to capture the context, the attributes of the objects are computed and change constantly. The key classes in the ontology are:

  • MediumTermContextThread - One of the user's current tasks, several user actions can be grouped to one such thread to express that the user is working on an ongoing task to which the actions and ContextElements belong. There can be multiple ContextThreads, when the user works on multiple tasks. As a contextual element can only belong to one ContextThread, the same resource may be in the attention of the user in two or more different threads. Then, two different context threads contain (via the contextual elements) the same resource. The normal case though, is that the user works in multiple, distinguished threads, and resources seldom overlap.
  • LongTermContext - the long-term attention model of the user on all resources
  • ShortTermContext - contextual elements that are in the short-term attention of the user

There can be multiple MediumTermContextThreads but there is only one LongTermContext (comparable to the long-term memory of the human brain) and one ShortTermContext (comparable to the short term memory of human cognition).

The current work context does not model geographic location or other physical properties of the user's whereabouts but this would be a good extension.

An attention level (floating point number between 0 and 1) is computed for each thing in the PIMO, expressing how much attention the user has given to the resource currently. Whereas currently means: related to the user's current task. In practice, the current task may not be known or not detected, so currently means the current work time (further simplifications are: today, the current hour, one session of interaction with the computer). During the current task, all user actions are recorded and the attention levels are increased for the touched resources. As in the short-term memory of the user, only a few elements can have much attention, so attention fades away over time.

By default the attention level is 0. Things with an attention level of 0 do not need to be represented in the current work context, they are not part of the "current work context".

The component receiving the observed user actions and making these calculations is now called Context Boiler, its functionality is Gathering and Eliciting Context. from Schwarz2005:

Adding the new native operation (nop) to the context means, first, creating a ContextualElement for the nop, and, second, raising a ContextualElementAddedEvent. This event will be catched by context elicitation modules. Taking the received user operations as evidences, more higher-level contextual information is estimated: Potentially relevant topics the user works on are inferred on the basis of recently touched documents. Additionally, a case-based reasoning approach estimates potential user goals and relevant workflow tasks.

Current Work Context

The current work context can be polled from the UserWorkContext service:

/**
 * retrieve the current work context, return only the elements 
 * @param threshold a number between 0 and 1, only resources with attention value greater than this will be returned.
 * @return an RDF/XML encoded representaiton of the current work context, containing only the 
 * resources in context and their attention values.
 */
public String getCurrentWorkContext(float threshold)

The current medium-term context thread can be retrieved, set, and changed via the service interface. You would typically change the context thread when you know that the user switches to a different work context, such as working on a new task.

Intended Usage of Current Work Context

Applications can use the current work context to assist the user, resources in attention are good candidates for annotations, default values, etc.

In Schwarz2005, examples are given what the context can be used for:

  • Assistant Bar: showing the currently active tasks, persons, topics
  • Annotating Current Documents: A Note-taking application allows quick and easy note-taking. The most important point here is, that entered notes are annotated with the user’s context. The accessibility of a note’s creation context enables (a) contextsensitive recherche and (b) context-triggered presentation of notes
  • Automatic Bookmarking: the browsed webpages are automatically annotated with the current work context, giving more retrieval clues

Implementing User Work Context

Details about the Context Server's internal implementation:

UserWorkContext service is here:

Current code of UserObservationHub is here:

Ontologies are here:

Storage of historical NOP data

The NOPs are stored in the "context" repository in RdfRepository. They are not stored in individual named graphs, because the performance of Sesame broke down when having many named graphs in this case, instead all triples are (as of 22.7.2008) stored in the default graph.

Components used

We use:

Other Existing implementations that were not used

from the EPOS and gnowsis 0.8.3 project:

from the MyMory project:

Loading historical NOP data

At the moment (July 2008) log data delivered by User Activity Logger are parsed and loaded to NEPOMUK's usercontext model. The data is from a five week user activity, size of the log file is 2.9 M and the resulting user contex model has 147k tripples.

How to load PAS logger data into NEPOMUK

Requirement

  1. UserObservationHub Project from DFKI.
    https://usercontext.opendfki.de/repos/trunk/UserObservationHub
    
  2. PAS logger data generated from User Activity Logger v.0.0.2 (WindowsLoggerHubPlugin does nnot accept log file from older User Activity Logger. It's easy to fix though. )
  3. de.dfki.km.usercontext and org.semanticdesktop.nepomuk.comp.usercontext are running in OSGi mode.

Steps

  1. edit or create UOH properties file

[UserObservationHub/etc/userobservationhub.properties] to adapt your PAS logger data location. e.g. windows_log_file = C:/Documents and Settings/$user_name/Local Settings/Application Data/User Activity Logger/t_log.txt

  1. edit or create [UserObservationHub/etc/lastLogLineRead.txt]

to line number in PAS log file you want start to parse. If you want to start from the beginning of the file put 0 in it.

  1. Run StartUserObservationLaunch.java

see http://usercontext.opendfki.de/wiki/FirstSteps

When you see something like this, then you are on the right direction.

StartUserObservationHub: starting
DragontalkListener is now listening (please check thunderbird/firefox dragontalk extensions):
    port    : 9996
    handler : UserObservationHub_MozUserObsApi

WindowsLoggerHubPlugin
  logFilePath (Activity Logger NOPs)=C:/Documents and Settings/$user_name/Local Settings/Application Data/User Activity Logger/t_log.txt
2008-07-24 16:24:10.509::INFO:  Logging to STDERR via org.mortbay.log.StdErrLog
2008-07-24 16:24:10.641::INFO:  jetty-6.1.5
2008-07-24 16:24:11.964::INFO:  Started SocketConnector@0.0.0.0:9995
REST user obs listener is now listening :
    port    : 9995
    url : http://localhost:9995/userobs

use commands like http://localhost:9995/userobs?uri=http://myuri&model=<xml></xml>
StartUserObservationHub: adding listener: OutputObservedNops. Logging NOPs on stdout
StartUserObservationHub: adding listener: LogObservedNops. Logging NOPs to logfile
  logFileDir (UOH NOPs)=/Users/bender/.UserObservationHub
StartUserObservationHub: adding listener: NepomukNopForwarder
Nepomuk connection is closed, trying to open it every 60 seconds
StartUserObservationHub: adding listener: RemoteUserObsListener(localhost, 9998, MyUserObservationHub_Listener)
Nepomuk connection is active
StartUserObservationHub: adding listener: RemoteUserObsListener("localhost", 11727, "UOHListener")
loading rules from jar.

Querying data

A typical SPARQL query, can be: "Give me all activated window on 15 April 2008 with a file attach on it, application name of the window, name of the file and accessed date"

PREFIX rdfs:    <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX xsd:    <http://www.w3.org/2001/XMLSchema#>
PREFIX nrl:    <http://www.semanticdesktop.org/ontologies/2007/08/15/nrl#>
PREFIX nao:    <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#>
PREFIX pimo:   <http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#>
PREFIX nie:   <http://www.semanticdesktop.org/ontologies/2007/01/19/nie#>
PREFIX nmo:   <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#>
PREFIX nfo:   <http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#>
PREFIX ncal:   <http://www.semanticdesktop.org/ontologies/2007/04/02/ncal#>
PREFIX nco:   <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#>
PREFIX nop:   <http://ontologies.opendfki.de/repos/ontologies/userobs/nop#>

SELECT DISTINCT ?date ?appName ?wDisplay

WHERE {
?s rdf:type nop:ActivateWindow .
?s nop:timestamp ?date .
FILTER(?date < "2008-04-15T23:59:59Z"^^xsd:dateTime  &&
?date > "2008-04-15T00:00:00Z"^^xsd:dateTime 
)

?s nop:window ?window .
?window nop:windowDisplays ?wDisplay .
?wDisplay rdf:type nop:File .
         
?s nop:application ?app .
?app nop:name ?appName .
}  
ORDER BY ?date

Related Work and Literature

UserWorkContext on KDE:

See also:

Last modified 6 years ago Last modified on 10/31/08 15:54:48