Approach

Visual Representations

Figure 1. The Theseus main display is comprised of several visualizations.

Theseus uses several different visual representations. The "Contact List" uses a tree view to group contacts by the projects in which they participate. Once a contact is selected, the visualizations will populate on the right hand side of the contact list. Alternatively you can search for a contact using the input text box below the words "Contact List."

Figure 2. The pie chart shows a developer's activity in multiple projects.

The Projects visualization (Figure 5) uses a pie chart to show a contact's activity broken down by the projects in which they participate. The percentages are based on amount of open work items assigned to the contact in question. Clicking on a slice of the pie displays the number of open work items that contact has in the project as well as work items across all projects that contact has resolved that were not originally assigned to them. This indicates "extra" work the contact is undertaking.

Figure 3. A linear scale shows how fast a developer's is to respond relative to the average.

The E-mail Responsiveness visualization shows the contact's standard score of their average reply time to e-mails using a linear scale. A standard score indicates how many standard deviations a data point is in relation to the average. Lower scores (left of the average) indicate faster reply time; higher scores (right of the average) indicate slower response time. Mouse over the blue dot to see the contact's exact reply time.

<
Figure 4. The time zone overlap shows overlap in the time of day between the user and another developer, as well as when that developer is actively sending e-mail.

The Time Zone Overlap visualization shows overlap in the time of day between your current location (Irvine), and the contact's location. It is composed of two separate parts: the time in the user's location (Irvine) is displayed to the left while the time in the contact's location appears to the right. The seven days of the week are shown with a one letter abbreviation on top of the visualization.

Each line indicates an hour of the day starting at 8 am at the user's current location. A green line indicates normal work hours, 8 am - 5 pm. A yellow line shows the time of the day where people are awake, but not necessarily at work, 7 am and 6 pm - 9 pm. The red line indicates sleeping hours, 10 pm - 6 am. A black circle on a line indicates that a contact is checking and sending e-mails during that hour.

Evaluation

We evaluated Theseus in the laboratory with 12 professional developers and 28 graduate students with experience developing software to test whether it increases trust, whether it is usable, and whether developers can make meaningful insights using it. The independent variable (control) in this experiment is the presence or absence of the Theseus tool. To prove the hypothesis, the tool must yield significantly higher trust compared with the no tool condition. Thus there are at least two different conditions: 1) no Thesus tool and 2) Theseus tool. Because attributions may be situational, dispositional, or a combination of both, condition 2 was further sub-divided into three conditions: 1) Theseus situational, 2) Theseus moderate situational, and 3) Theseus dispositional.

Figure 5. Trust scores were significantly higher when using the Theseus tool compared with not using the tool (a-b comparison). Scores range from 15 (low trust) to 75 (highest trust), with a neutral score or midpoint of 45.

Figure 6. Attributions were marginally significantly more situational when using the Theseus tool compared with not using the tool (a-b comparison). Scores range from -7 (highly situational) to 7 (highly dispositional), with a neutral score or midpoint of 0.

Figure 7. Box plot for usability scores in each category. Scores for 'Usefulness,' 'Information Quality,' and 'Interface Quality' were out of 5 points, with lower being more usable. 'Overall Usability' (far right) is the sum of the three category scores (lower is better).

Figure 8. Participants were able to become quickly immersed in the information visualized by Theseus.

Findings

1. There is significant evidence that Theseus results in higher perceived trustworthiness compared with not using the tool (Figure 5).
2. There is marginally significant evidence (p=0.052) that Theseus results in more situational attributions compared with not using the tool (Figure 6).
3. Developers agreed that overall, the tool's usability was high (Figure 7), but it could be improved with a better mechanism for comparing more than three developers at once.
4. Developers were quickly able to become immersed in the data provided by Theseus (Figure 8).
5. Developers made few incorrect observations using the interface.
6. Developers tended to give situational attributions when having no information about or no previous working experience with a remote developer.