Feedback Needed: Interactive network diagram demos

8 August 2012 28

This post lays out some of the importance of interactivity for network diagrams and we think is well worth a read (or at least a skim!); however, if you wish to skip to the demos, these are linked at the end of the post.

Visualizing network data is a prime example of a research activity where both desktop tools and online libraries exist, but are not well connected with one another. Desktop software like NodeXL and Gephi provide the ability to collect and visualize network data (users on Twitter, videos on YouTube, etc.). These applications, however, provide limited facilities for hosting, sharing, and optimizing visualizations for the web. Numerous technologies, frameworks and platforms exist for displaying network data online, but they often depend on bespoke data formats or a high degree of technical expertise.

The visualization below shows the Twitter network of the Oxford Internet Institute (@oiioxford), and was produced without a single line of code needing to be written. Data was collected through NodeXL and visualized with Gephi.

Twitter Network of @OIIOxford

However, when trying to share this visualization online, the software we had used to create it left us wanting.  The only option readily available was to export as an image and post it to a blog, department website, or gallery. There are numerous practical reasons that this solution was undesirable. For example, it would have been necessary to export at an extremely high resolution to preserve the legibility of the smallest nodes, but this would have created an impractical file size. Also, navigating such a large image file in a web-browser would likely prove awkward and difficult to manage for users, which would limit the intelligibility of the visualization. However, perhaps a more pertinent issue concerns the general usefulness of static network visualizations. When viewed in Gephi or NodeXL, the easy accessibility of node and edge attributes encouraged the data to be explored. These attributes are flattened and reduced to purely visual indicators when a static visualization is created.

Network visualizations like this one seem a prime candidate for interactivity, and indeed great work has already been done with the Sigma.js JavaScript library to enable the rendering of network graphs with HTML5 canvas. This library allows all newer browsers as well as tablet devices and phones to display the visualization without any extra plugins (e.g. Flash) or software (e.g. Java) needing to be installed.

Sigma.js does, however, require knowledge of JavaScript and HTML to produce an interactive network diagram. The amount of knowledge required grows further if the creator of a visualization wishes to provide an infobox with more node information, an ability to search for a specific node, or to change fonts and colours. The library also could be improved in some subtle, but important, ways: we’d like to add the ability to detect when the user positions the cursor over a specific edge, properly handle redrawing on tablets when the visualization is zoomed, and provide the option for edges to be a blend of the colours of the two nodes they connect. We will work to make these changes and provide a hosted wizard to allow a user to easily create and customize the visualization (what attributes of a node should be shown on click, what title and explanatory text should be included, should any edge information be shown on hover, should a search function be included, etc.).

Before the wizard is released, however, we are very excited to release two demo visualizations as examples of what could be produced and customized with the wizard. We’ve tested these visualizations in the latest versions of Safari, Firefox, Opera, and Chrome on Windows, Linux, and Mac, but we consider these to be very much work-in-progress. We’re releasing them to get feedback that we can incorporate into the project. The more specific you can be in describing any suggestions or issues the better. Please at a minimum include your operating system (Windows, Mac, Linux) and browser version.

So, please leave feedback! What works, what’s missing, and what would you ideally change? Leave a comment below or send an email.

Thanks so much!

Demo 1: Visualization of followers at @OIITwitter

Demo 2: Visualization of UK Central Government

28 thoughts on “Feedback Needed: Interactive network diagram demos

  1. Pingback: Feedback Needed: Interactive map demo | Interactive Visualizations

  2. Ron Newman

    How do I say this without sounding self-serving? Maybe I can’t, so I’ll go ahead in the interest of ideas and improvement. MyIdeaTree.com does most of what you are requesting with the exception of two things:
    1) import from NodeXL – actually, this was suggested to me last February and I’ve just been putting it off, while thinking on whether to do it or the .dot format.
    2) large graphs in a browser – I’ve limited graph sizes to what is easily visible in a browser and because large graphs are computation-intensive. But since I’m on a grid system that may not be the big deal I think it is. If anyone would like to experiment I’m willing to give it a try. If it is, installation of MyIdeaTree on one’s own server would solve that issue. I’d need to enable zoom.

    All the other stuff you mention, no Flash or Java, colors, mouseovers, shared interactivity, export to zoom-able,clickable image (actually a .pdf), HTML embedding, programmatic graphing via API, etc., are either already there or available in the underlying system but not exposed yet in the UI for reasons of keeping the UI simple. One difference: your demo opens a sidebar when clicking a node, M.I.T. opens a new window with whatever URL is associated with that node.

    Other things are possible, like layered maps. Just waiting for someone to ask.

    Reply
    1. Scott Post author

      Thanks. Hadn’t heard of IdeaTree — had always used FreeMind on the desktop. As a mindmapping tool it seems to be in a slightly different class from displaying larger, messier empirical data we’re working with. Thanks for pointing it out, however.

      Reply
      1. Ron Newman

        Yes, this is a common misconception that I encounter. Mind mapping tools like FreeMind, etc. are a different class from programs meant for fully connected, non-hierarchical network graphs, like MyIdeaTree.

        Another differentiation between the two is the method of sharing. Most do so by exporting files, but these can get out of sync. MyIdeaTree shares in real-time, i.e. a map can be edited simultaneously by an invited group.

        Anyway, I think we’ve got something here that has already solved most of the problems you mention. Would you be willing to verify this and potentially work with me on moving it forward?

        Reply
    1. Joshua Melville

      Thank you for the feedback Peter.

      We are putting the final touches to the configuration file system which we hope will let advanced users tweak their layouts independently of the wizard. We will be sure to write a post on this at a future date.

      Reply
  3. Angela

    Love the idea of a wizard for interactive network visualizations! If you’re really interested in “what’s missing”, my requests would be:

    – temporal controls. (cannot tell you how many times the need for networks that evolve over time has come up in my work. it affects the data itself and requires some thought about “freezing” the position of nodes over time, so of course it’s not a trivial problem.)
    – a chooser for other layout algorithms. why evenly spaced nodes? the example networks have group and link (at least “presence of link”) info; why not position nodes with a force-directed layout, or at least give the option? real world data sets may not have category information for nodes, so seeing the visual space between clusters becomes much more important. even spacing seems to be a new trend, but i don’t know if it scales well.

    Perhaps those are gaps of sigma.js and not within the scope of this wizard, but that’s where I think development should be heading for network visualization tools.

    Thanks! Nice work so far.

    Reply
    1. Scott Post author

      I am really interested and appreciate the honest feedback. I fully agree on temporal controls. This is something I want to address, but it won’t fit within the current project timescale. I’ll probably keep doing some work on this personally, however, as I’m working on a related project with 15 years of Internet Archive data and want to try to show the time dimension of this in some interactive way (http://www.oii.ox.ac.uk/research/projects/?id=88).

      Right now the layout algorithm is fixed by the creator. For the short term, we’ll ask the user to layout the network in desktop software and then provide the datafile and visual display customization information in the wizard. Probably these two demos are just bad examples for force-directed layout. In the case of Twitter we pared the graph down to mutual relationships and only are showing the largest strongly connected component. We tried a few force directed layout algorithms but they didn’t really separate the nodes much. A Facebook network (created with NameGenWebDev http://apps.facebook.com/namegendev/ ) we tried separates into clusters well, but we didn’t publish the network as it seemed a bit too personal.

      Reply
    2. Joshua Melville

      Thank you Angela for your feedback! I agree that the visualisation of time-series data is a highly interesting and increasingly relevant problem within SNA. Emerging approaches in the literature are quite computationally intensive at present, and will (I believe) increase in quality dramatically over the coming years. As Scott mentioned, we made a conscious decision for our tool to not deal with graph layout for two reasons: 1) They are computationally demanding (my NameGen application uses Gephi toolkit server side for layout, and small networks still take over 30 seconds to converge – we simply do not have the resources to deploy at that level) and 2) As your comment about force-directed shows, decisions about which layout is “best” for a given situation are non-trivial and indeed highly personal. Different families of layouts would require different sets of parameters that a user could tweak, and we felt that this would be both programmatically and conceptually difficult. Though our toolkit aims to make interactive visualisation easier, we still believe that for a visualisation to be truly informative/useful, the researcher will likely require some knowledge of the core concepts of social network analysis, of which graph layout is one.

      As an aside, the twitter network is actually using a force directed layout (force-atlas 2). The even spacing occurs because we used the option to avoid node overlap.

      Reply
  4. Anna

    Nice work!

    The problem with sigma.js is that it doesn’t work in IE8, out of the box – would be great if you could add excanvas support. I haven’t tried the demo in IE8, so apologies if you’ve already done that.

    Is the code-in-progress available? And will the finished tool be available under an open source licence?

    Reply
    1. Scott Post author

      Thanks, Anna for the feedback. We’ve included the excanvas tag on the page, but haven’t tested it much (at all?) yet. This is definitely on the todo list before the end of this month.

      Code is on github
      http://www.github.com/oxfordinternetinstitute/InteractiveVis
      and will be under an open-source licence. We’ll have to review some of our dependencies (fancybox, jquery, sigma.js) to figure out exactly which license.

      Reply
    2. Joshua Melville

      Hi Anna. Thank you for your feedback!

      It is true that sigma does not currently work with IE8 because (as you rightly point out) it does not support the canvas element. As Scott mentioned we are making attempts to support other browsers using excanvas, but it may be that this is one of those situations where we must break some eggs to make an omelette. Our project aims to use open web standards to ensure that these visualisations are compatible with all devices that adhere to the principle of openness and standards compliance. A solution using excanvas may be impractical for performance reasons, leaving us requiring the use of flash which would contradict this ideological position. One idea we could consider would be using browser sniffing to (for example) prompt users of older/incompatible browsers to install chome-frame, or upgade to newer versions (the latest version of all major browsers, including IE, works with sigma). We are aware that at present the visualisations do not always fail gracefully, and would welcome any technical expertise or advice in this area.

      Reply
      1. Scott Post author

        The other problem (even with IE 9) was that it wouldn’t correctly parse the XML in the gexf files. We’ve worked around this now as explained here:
        http://blogs.oii.ox.ac.uk/vis/?p=149
        It would be great to get feedback from others with different versions of Internet Explorer.

        Reply
  5. Scott Post author

    Received a comment outside of the blog that we should allow the viewer of a visualization to change the attributes mapped to node size and/or colour (e.g. let the user select if node size is base on betweenness, degree, etc.). The creator of the visualization could set the available options (or disabling the feature entirely). Just posting this here for future reference, but also feel free to comment if this would be useful for others too.

    Reply
  6. Alberto Cottica

    Well done! I concur with the temporal control request.

    BTW, most developers I talked to prefer to work with another library called D3 (http://d3js.org/). It does virtual physics, draggable nodes and all sort of cool things.

    Reply
    1. Joshua Melville

      Hi Alberto. Thanks for your input!

      We considered D3, along with many other popular JS libraries that are capable of handling network data (please see our previous posts on the the foundations of the project), even going as far as developing prototypes with them.

      We concluded that for our purposes sigma was a better option for a few reasons: 1) Though D3 can use canvas natively (or any rendering system) this must be specified and developed independently. Canvas support is not a core feature of D3, and has not been packaged together with an interface specifically designed for the manipulation of network data. D3 is extremely versatile, but we felt that this came at the expense of specific features for network data, and specific optimisations for rendering to canvas. 2) Because we have limited resources and timeframe, we wanted to avoid “reinventing the wheel” as much as possible. The fact that sigma is already using canvas and is geared up for network data specifically means that we can spend more time developing other aspects of the interface. 3) The interesting features of D3 that you mention (dragability and realtime layout) could be implemented in Sigma should the need arise. 4) Those features are not presently being considered for inclusion with our toolkit (though we would welcome feedback on if users believe these would be valuable). Please see my comments further down this page for more background on why we have chosen not to do graph layout in the browser. Suffice to say, performance remains a big issue, and will likely remain so until WebGL is a viable option. 5) We don’t intend users to be involved in the behind the scenes operation of the visualisation, since this tool is aimed primarily at those who would otherwise be unable to create an interactive visualisation of their data. We imagine that those users advanced enough to have some knowledge of D3 would have no trouble creating the sorts of visualisations our tool produces for themselves.

      I hope this makes our reasoning a little clearer. Believe me when I say this was a tough decision to make!

      Reply
  7. Pingback: Network visualization lunchtime talk | Interactive Visualizations

  8. Pingback: Interactive Maps | Net Increase?

  9. Jia

    First thank you so much for making this tool for doing exactly what I have been trying to do on my own without much success – putting a giant Gephi network graph online.
    Can you point me to a place where you go through in a bit finer detail how to make the group selector, as well as adding individual node info to the information pane?
    It is all in the first example of the twitter network of @OIIOxford.
    Thank you.

    Reply
  10. Scott Post author

    Hi Alice,

    Thanks for writing. These are both excellent suggestions and something I’d like to do. I don’t unfortunately have an expected time frame for completing them at the moment. As a work-around for #2, you can save a copy of your project in Gephi and delete whatever attributes you don’t want shown before exporting the graph. I know this is a pain, but hopefully it can still lead to the results you are seeking.

    If the group selector box could be personalized, where would you want the labels to come from ideally? A node attribute or separately type this list in?

    Cheers,
    Scott

    Reply
  11. Ramkumar Gokarnesan

    Hi,
    Are you still actively developing the library/ the Gephi plugin?
    -Ramgo

    Reply
    1. Scott Post author

      Hi Ramgo,

      We’re still developing but slowly and intermittently as a side project. I’d very much like to get the plugin moved to the new version of Gephi, but haven’t had a solid block of time to do so yet.

      Best wishes,
      Scott

      Reply
  12. sam

    Hi, I would love to recreate what you have done with this demo, the interactive graph with the search ect, but on my own network, how do you recommend I go about doing this?
    Thanks

    Reply
    1. Scott Post author

      If you can get your data into Gephi (a free network visualization application), you can install the Sigmajs Exporter plugin, which will produce a visualization very similar to the demo customized as you like.

      Reply
  13. Anirban

    Hello, this is excellent work.
    I have an issue though. I exported the network webapp, however the nodes attributes do not show up. It shows the numbers instead. I tried selecting/unselecting the replace nodeids with numbers.

    Reply
    1. Scott Post author

      Hi Anirban, I’m not sure what you mean by the attributes not showing up. They should appear in the right side bar when a node is clicked. If you can link to an example where you’re having the issue that would be helpful to see what is going on.

      Reply
  14. Hafidz

    Great tool! Have been playing with it for hours now.

    Does anyone know how to add additional group selector however? i.e, to filter the network by modularity, eigenvector and other parameters?

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *