Category Archives: Semantic Web

The Mythology Engine

3 Replies

Finally, after months of writing, thinking and beavering away with the good folks of the BBC’s R&D Prototyping Team, we can reveal a prototype featuring many of the concepts I’ve been discussing here. Thanks must go to the great team there, who’ve helped turn these ideas into some form of reality. Now, I’ll be doing what I can to take things onto the next level, and get something fully built. I’ll blog more about the development process of the prototype and what it’s capable of, on the weekend, but until then, here’s the official BBC blog post: Mythology Engine on BBC Blogs

On Avatar, 3D, Augmented Reality and Truly Interactive Television

3 Replies

Happy New Year! Firstly, a little apology – I put a note at the top of my last blog post saying that I’d explain my use of the terms ‘Internet’ and ‘Web’ soon – I did in fact write a post – but somewhere along the line it never made it out into the wild world. So for that, sorry – but the gist of it was that perhaps I should have used the term ‘Internet’ to refer to the underlying infrastructure network, and the ‘Web’ to refer to the network of information that can be built on top of this.

And now on to the main topic for today. I’ve talked previously about how we could/should be using the Web to provide representations of the narratives we currently tell via radio and television. I said that whilst on-demand services such as iPlayer have had great success, and have certainly improved the consumption of media, they’re not really game-changers, in that they are an attempt to replicate the form of a linear medium within a non-linear medium. As such, although they benefit from the latent abilities of the Internet (speed, distribution, on-demand), they do not take full advantage of the Web. These are still TV or Radio ‘adaptions’ of stories, being distributed by the Internet. What we need is the ‘Web’ adaption of the same story.

I’ve been working over the past month or so on a prototype that will explore these possibilities. At first glance, it seems to be very similar to Wikipedia, in that there are pages for characters, places, events, and links between them. The audience can undertake similar journeys to that of a visitor to Wikipedia – i.e. non-linear, explorative journeys – things which people are already doing (for instance when they say they got ‘lost’ on Wikipedia – in a positive sense!). However, what is different is that these URIs, and the HTML representations of their subjects, are connected directly in the same way as the story itself is being told. Thus, a collection of these URIs, joined together through hyperlinks, can be seen as a small web, or constellation, representing the story itself – a Web adaption – which allows the audience to explore the story from all angles, and gain new perspectives.

For a while now, I’ve been thinking of how the mainstream user will benefit from all this. I think the effect will be fairly subtle at first, and I was imagining two ways of experiencing the story – firstly by hopping from URI to URI and being immersed in the ‘world’ of that thing, and secondly, by taking a step back and examining the web of connections between things, and travelling through this web along a particular path – the act of telling the story. The latter, I imagined, would be through the form of some fairly standard ‘dots and lines’ visualisation, but at the back of my mind, I wasn’t satisfied with this. Co-incidently, I then saw Avatar in 3D at the cinema. Personally, I felt pretty let down by the paucity of imagination shown in the storytelling, but I had to admit that the 3D effect was intriguing. Perhaps, rather than visualising the links through a limited, 2D ‘dots and lines’ diagram, the audience could gain a greater understanding by viewing the story’s Web in 3D, allowing them to see all sides of it.

This is still a possibility, though again I’m very aware of the lack of availability of devices and technologies in the consumer market which support 3D. That, of course, may change, but I wondered whether there were other ways of improving the experience. I was worried that without this, it would just seem, to the general audience, like a replication of Wikipedia (albeit containing information that neither Wikipedia nor fan-wikis hold in such a structured, clickable manner).

And then I considered the ideas of convergence and Augmented Reality – essentially reminding myself that the Internet and the Web, and our interaction with it, need not be restricted to the browser. The Web is, at its heart, merely the highly structured data store – on top of which we can build user interfaces across virtually any connected platform. So, I started to think about mobile and TV viewing. When I’m watching drama, or sport, or the news, I often want to know more – why something is important, what someone is referring to, more about a player, what’s the bigger picture etc. At present, the content is communicated to me via the screen, I interpret it, and then have to go off on my own search to find out more. When doing so, I have to begin again from scratch, communicating the same content (or a near approximation of it) with a computer connected to the Web. What if the content presented to me on screen also had the underlying semantic structures that meant it could do the communication with the Web?

The simplest form of this would be on a mobile device, where, whilst watching the programme (and indeed at any other time), you would navigate to a portal which can guide you to the correct URI contained within the narrative structure – this could take the form of a search engine, or a listing – that way, I could search for ‘Jack Bauer’ and be taken straight into the ‘world’ of 24 – or, more powerfully, if I witnessed an important event happening on screen, I could click the relevant link in the portal, and see other events that have led up to this, more information etc.

But there’s an even more advanced version of this, which I strongly believe could be prototyped and developed pretty quickly. There are technologies available which can take a drama script, and output RDF triples, creating Web structures which represent every element of the narrative, down to the words. These can also be enhanced by matching the triples to timing information within a media representation – so, for instance, identifying that an event happens at 20 minutes into this particular version of the episode, but 15 minutes into another version.

Couple this with the growing links between the consumption of media and the Internet – TV over IP, such as BT Vision, or even on-demand services such as iPlayer. The media is being streamed to the audience – but this is potentially a two way channel – and if we have all the information about the narrative structure and timings for the programme available on the web, then the user can access that wealth of information whilst they are watching – either directly onto the screen, or on a supplementary mobile device.

Just think of what this means for drama, for starters. The ‘flashback’ device in storytelling, essentially used to give the audience a reminder of previous events, so that they can greater enjoy the current story, no longer needs to be incorporated into the linear representation of the story – because as the story is produced, it is connected on the Web to all previous parts of the story. Thus, if the audience wishes to learn more about something, or get a reminder of previous events, they can access them. If you were watching a programme, you could pause it, or activate your mobile device – the playback device would know the timing information of the audience’s action, and could query the Web to find the relevant URIs of information, and present the knowledge found there, back to the user. For instance, if a reference was made on-screen to a past event, rather than the production team having to add in a flashback sequence, the audience could activate the communication at the point of reference, and be presented with the original clip of that event happening. Taken further, this then starts to really break down the linearly-imposed walls between ‘episodes’ of programmes – which are, of course, relics of the original linear nature of television – and instead presents the audience with something much more suited to their own mental models of the narrative they are consuming. In the end, it wouldn’t really matter what episode you were watching – you could be freely exploring the whole universe of narrative, surfing between clip and clip, consuming the story in the order you prefer. Obviously it’s not something you’d want to be doing constantly, but it brings the freedom of the Web to the self-imposed closed structure of the television – and opens up whole new ways of experiencing the stories we tell.

As I mentioned, these are only fresh ideas being formed as we speak, so I’m sure the solution isn’t completely straightforward, but it really does seem that all the various puzzle pieces exist, they just need to be brought together – and the potential could be huge.

The Fourth Medium

1 Reply

Stories as TV Programmes, by tristanf

It’s been a while since I last posted. Things are going well, and I’m glad that all this has come on leaps and bounds since this time last year. Hopefully, by this time next year, there will be something practical to show for all of it – stay tuned, and keep an eye here if you’re interested in the Ontomedia work.

I’ve been busy refining the ideas that I’ve discussed here, experimenting and talking to others about it. In the course of doing so, however, I’ve felt that there’s one major part of what I’m trying to put forward which doesn’t always get through. It’s something I touched upon in the last post, but I thought it might be good to expand and explore again here.

In the case of drama, the writer is the starting point for everything. Their ideas and the ways in which they are communicated through their writing are central to their craft. The story comes first, and then it is translated and adapted to suit a number of different media. There are, of course, three main types of media that I’m referring to – text, speech and (moving) pictures. This then roughly translates to the medium of print, radio and television respectively. The audience for the writer’s work is implicitly aware that they are experiencing the story through the filter that the medium overlays on the narrative. In the mind of the audience, then, they must decode the filter in order to get to the real ‘prize’ – the narrative, the context, the ideas being communicated.

This isn’t to say that the filtering effect of the medium (be it text, speech or pictures) is necessarily a negative influence. Careers and whole industries have been built on the ability of people who understand the natural tricks of the medium, and are able to utilize them to create an effective and compelling representation of the story. Equally, tricks of the trade can be noted and enjoyed by audiences who find comfort and satisfaction in familiar practices.

Of course, when constructing a narrative, the creator is likely to be aware of the conventions, limitations and advantages of the medium they have chosen to communicate through. This may indeed shape the narrative itself, but it remains true that you should be able to strip away the medium’s influence, in order to access the ‘pure’ story at the heart of the work. This can then be re-contextualised through a different medium – some of its beauty in the former medium may be lost, but a new perspective may reveal new highlights in the new medium.

OK, so far, so (fairly) obvious, if a little abstract. The point that i’m trying to make is that when you use a medium to construct a representation of a narrative, the medium plays an active role in the communication of that narrative to the audience. Or at least, it should do, if the medium in question is used to its full potential.

So, with that in mind, let’s examine the media that we’ve discussed. We’re all aware that print, radio and television (alongside film) have their own techniques, advantages, quirks. Writers, producers, technicians, artists, all use these quirks to make representations of a narrative in a way which is best suited to their chosen medium. That is to say, you have the book of Swallows and Amazons, the TV adaption, the radio version, even the film. Note also the language being used – adaption, “…brought vividly to life…” – each acknowledging the fact of the different media being used to tell the story.

However, then we come to the Web – and what do we find? It would appear that, from current use, we cannot call the Web a fourth medium. Instead, it is merely a distribution channel for the other three, akin to a TV set, a radio, or a bookshop or printing press. Not that this is a bad thing in and of itself – applying the technology of the Internet – of a network – in this way is pretty obviously a good thing, despite the legal ramifications – communication has never been so cheap and widely distributed. But earlier I said that representations of a creation through a medium involve some form of adaption, not just delivery.

The book, the TV, the radio, the film – these are all isolated, one-shot representations of a creation. They are produced, packaged, sold, distributed, consumed – in isolation. In order to remix, re-adapt, and so on, we either have to go directly back to the source, to the creator, and ask them to rework their creation, or we have to cut and paste, knowing that in doing so we are left with the pieces of the story still in situ of the media from which we took them – they cannot truly be extracted.

However, the Web is different – it has the potential to be the fourth medium, a networked, connected medium, thereby very different from the other three. Rather than sound effects, paragraph breaks or camera tricks, the Web has its own natural tricks and advantages – the network, the connections. These, of course, are the same connections that we all make when consuming narrative from the other three media – but rather than being isolated within us, the Web allows us to expose these connections, explore them, analyse and create new links, enabling us to enjoy narratives and stories in a way which is both natural to the Web, and, more importantly, natural to our minds.

Just as we take a story and adapt it for a book, for TV etc, so we should adapt stories for the Web. We should forget about scenes and chapters, and concentrate instead on the story itself – the events that take place. In this way, we can begin to construct entirely new adaptions and representations of stories. Only then can we say that we are using the Internet to its full potential, and only then will we have added a fourth medium to our repertoire of cultural and creative expression.

The Web as a Creative Tool

4 Replies

Radio Daze, by Ian Hayhurst – creative commons license from Flickr

This year, the World Wide Web celebrates its’ 20 anniversary. I’ve been thinking about what use we’ve made of this technology during this time, and the way in which I like to think about how we could help fulfill the potential of the Semantic Web. As ever, thanks go to several people and bloggers out there whose ideas have inspired me, and the conversations I’ve had with them in general. Again, this may be stating the obvious to some people, but I feel it’s important to try and draw the threads of the patterns I’m seeing together in the hope that it will help others make the leap to looking at the potential of the WWW in a completely new way.

Up until recently, the Web has mainly been used as an enabling technology. By this, I mean that it has allowed us to do things faster, easier, cheaper, wider and for longer than ever before. However, these things that we’ve been doing with the Internet are, in the main, very much things that we were doing before the Web hit the mainstream. If anything, we’ve concentrated mainly on transposing these traditional methods of communication and interaction onto a new platform – and a platform is exactly what we’ve been using the Web for. Essentially doing the same as we’ve been doing before, but “now with added web!” as it were. On the way, we’ve (possibly accidentally) created concepts that didn’t exist before – for instance the whole notion of websites, but overall it’s been a case of doing the traditional things, using the web – and the benefits listed above have come as a kind of side-effect.

The industry in which I attempt to make a living – television and radio – is doing the same thing. The on-demand products from all the major UK broadcasters offer the benefits I’ve mentioned, but when it comes down to it, they are still just replicating traditional platforms – it’s all about using your device as a substitute for your radio or your TV. Yes, we get the extra benefits of stuff being available for longer, and potential personalisation, but we still haven’t fully escaped the mindset of using the Web purely as a platform – as if it was just a new type of box for watching or listening. It can be, but it can also be so much more.

The other way in which we have tended to use the Web has been as a commercial tool. I mean this in two ways. Firstly, again, it’s a case of imitating existing processes on the Internet – hence the success of retailers, such as Amazon and Play. However, I think that the main use has been the way in which the Web has been seen as vital to a commercial strategy – not just in terms of selling goods directly through the Internet, but in a promotional sense as well. If you want to be successful now, you need a promotional presence on the Web. Again, however, what role is the Web playing in this, apart from the side effects? Very little. We’re still promoting and distributing things, it just happens to be a new platform for doing so.

I think it’s especially interesting to note the dichotomy between the way in which we’ve transposed old methods of ‘doing’ onto the Web, whilst creating new ‘things’ which are ‘of’ the Web. But the crux of the matter is that we’ve never really (in a mainstream sense) tried to properly transfer the traditional ‘things’ which make up our world onto the Web, and then set about creating new ways of interacting with these things via the Web.

So that’s what we’ve done up until now. But what of the future?

In the mid-1990s, when CD-ROMs and the Web were beginning to puncture mainstream consciousness, the buzz-word was ‘interactive’. Yet I think this has always been a mis-nomer. The way we’ve used the web so far in terms of the creative arts still conforms to a flat structure. People create things, whether they be songs, pictures, television/radio programmes, even blog posts, post them on the Web, and that’s it. The thing that has been created is effectively fixed, static. Other people can create their own works as a result of these things, but again, it’s almost as if they perform the act of creation ‘offline’, and then only when it comes to ‘publishing’ does the thing go ‘online’.

The Web, as I understand it, is essentially very simple. As I’ve mentioned in other posts, it’s just dots and lines. The dots are the things we identify, and the lines are the links we make between them. So far, we’ve concentrated on using the dots to represent ‘pages’, and the ‘lines’ have been (mostly) simple navigational links, with little meaning invested within them. These dots and lines, though, could be used as a model for almost anything – they are, after all, the essence of communication, the construction of a narrative. What we should be doing, is using the Web in the same way we would write a book, or make a TV/radio programme.

By this, I mean that just as you pull together ideas, resources, things in the construction of a work, we would use the Web to do the same thing – except we’d be able to retain the links back to where the individual parts of the work came from, with less need for someone to do the hard work of analysis for us. For instance, knowing that a line in a TV show is a reference to a famous film from the 50s, knowing that an author is alluding to a Norse legend, knowing that a piece of music is sampling others, even knowing that a work of art was painted using oils or watercolours, or uses symbols which have distinct meanings – all this would be explicit and available to anyone, via the links – encouraging learning, truly ‘reading between the lines’, as it were. Indeed, then we could claim to use the word ‘interactive’ properly – because a work that is published by someone would no longer be a flat, finished structure – audiences would be able to explore it from all angles, trace links to other things, and, importantly, then create their own works by linking things together in a brand new way.

Of course, one of the main objections to the trend of making things available online is that we lose the context of things, the author loses the power. I think that I disagree here – that’s not a failing of the Web itself, it’s a failing of our limited use of it. If we were to use the Web in the way I’ve talked about, then authorship would be another valid link to make – and one that should always be traversable – credit would actually be easier to give, and would also hopefully, importantly begin to encourage a true breaking down of the walls between ‘producers’ and ‘consumers’ – we would, and should be, enabling the audience to create entirely new things, using our things – that’s still a valid thing to do, as long as the credit is given, and the links between what one person has originally made, and someone else has remixed, are made.

As I mentioned towards the beginning of this post, it’s almost as if so far, we’ve done things the wrong way round – we’ve been so busy creating the new platforms and enablers that we’ve failed to see the true potential of them – and that can only really be achieved once we start migrating not the processes and devices (e.g. the process of shopping, the ability to watch TV etc.) but the things we create for those processes, onto the Web. Dots can be more than webpages, lines can be more than navigational links. Create the things (from which we may create exciting new things we haven’t even thought of yet…), then refine the processes that help us find, share and experience them.

Narratives and the Semantic Web

3 Replies

Super Bowl Sunday Crystal Ball, by Circulating, from Flickr, Creative Commons license

“People assume that time is a strict progression of cause to effect – but actually, from a non-linear, non-subjective viewpoint, it’s more like a big ball of wibbly-wobbly…timey-wimey…stuff.”

The websites that we create around the narratives we tell currently focus on the objects within those narratives, like the playing pieces in a set of toys. More often than not, these are hand-crafted, static pages about certain editorially defined objects. Although we can record the links between objects implicitly as things that the audience can travel along, we tend not to expose them as things that the audience can explore and see in context. This, however, is what we are really interested in when describing narratives or telling stories. We define the objects within the world of the narrative, and then describe the interactions and changes between the objects. The intriguing thing is not necessarily the objects themselves, but the ways in which they change, or otherwise. A truly engaging website would therefore allow the audience to explore the world of the narrative not only by navigating between the objects, but by exposing and analysing the links between them, in order to derive more satisfaction.

Outside of the web, when we focus on an object, our minds give it context, and naturally establish the links between relative objects. For instance, when on a train journey, if I look out the window, I can see that branch of that tree which is placed there. We are instantly aware of both the object and its context, the thing and its links. Do the same thing with a computer, and it could identify and create a URL for a branch of a tree, but this would exist in a vacuum. It is up to us to give it the context. Using the principles and technologies underlying the Semantic Web, however, we can start to embed the context, the links, the meaning, so that, when using the web, we do not have to define these things every time. Instead, we can concentrate on uncovering and analysing those links, so that we can derive greater understanding and enjoyment from them.

Currently, websites such as www.bbc.co.uk/programmes define the objects, their contexts and links in a semantic web fashion, so that we can uniquely identify a particular object. Essentially, it provides the building blocks upon which we can establish the type of website I hae described above. Unfortunately, as far as I am aware, these building blocks are the limits of what we can currently, reliably, achieve. Emerging technologies such as SPARQL and RDF/graph visualisations will help us to build upon these blocks, but I do not think we currently have an established, reliable ‘toolkit’ or process that we can use to do this. However, this does not mean it cannot be done – it needs further experimentation. In the meantime, we can set about ensuring that the websites we build now will allow us to achieve the ideas mentioned above.

In the context of the BBC, there are two areas in which I can imagine the benefits of such an approach. The first, I will only give a overview of, as I have only thought briefly about the possibilities. The other, regarding fictional narrative, has been the focus of my previous blog posts, and I will continue the discussion here.

The first area is sport, particularly football. The BBC Football website contains a wealth of information, covering what is, in effect, the (almost) closed-off world of football. Fans essentially are following a narrative which spans matches, clubs, leagues, seasons, cup competitions etc. There is, obviously, some organisation taking place on the website – organising the clubs into their leagues, for instance. However, the links between these things – and here I mean not just the clubs, but the players, the action – are rarely revealed. We know that a team is relegated from a division because on one day their page exists within the ‘Premier League’ section, whereas the next, they are in the ‘Championship’ section. Their history may be recorded on the team’s page, or preserved in the numbers of a league table for a particular season, but there is no way of effectively (and, most importantly, engagingly) charting their fortunes. Of course, we can present these things in the numbers and bar charts and graphs, but they do not take advantage of the existence of the narrative behind them – which is really what people are interested in. Similarly with players. When two players go in for a tackle, we know that they have a history of confrontation, or perhaps an embarrasing own goal incident – what if we could provide the context around that tackle as and when, and after, it happens – filling in the back story, and getting the audience excited and engaged.

Similarly, by identifying and putting objects and events in context, we can give the audience something to latch on to. Take, for instance, a penalty incident. Say that the match was being covered on 5 Live with a commentary, it was shown and discussed on Match of the Day by pundits, and then also talked about on forums and 606 by fans. If we had an identifiable ‘hook’ for the incident, then potentially we could build a page which brought together all these different interpretations and discussions of the same event. That way, the audience would have an effective overview of the incident, with informed (and perhaps ill-informed!) opinions – their understanding and enjoyment would be enhanced, and of course, they could make their own contribution.

Back to the fiction – in my last post, I linked to a couple of images within which, I tried to explain what I aim to achieve, and where the benefits could be found. The first diagram establishes the episodes as a whole, regardless of series – and then drills down to a particular series, and a particular episode. A website that deals with a fictional narrative needs to remember that the episodes are merely a window onto the universe for the audience. If we intend to allow the audience to fully explore the universe, then apart from pointers leading them from/to episodes, as a form of ‘way-in’ (which, incidentally, should probably be through /programmes) the episodes themselves should (probably) not be included – all that exists are the objects (the places, the times, the characters) and the events.

The first diagram, once an episode has been specified, identifies the characters and events within the episode that are crucial to the narrative. For this, I limited myself to a handful of events and characters, which meant that I did not fully get the richness of the narrative across. However, potentially, we could identify as many events etc. as we require. Below the timeline of events (as presented to the viewer) there are coloured blobs, representing the characters in the events. This view shows us how the characters come and go throughout the episode (for instance, the Doctor only really appearing at certain points in the beginning, middle and end).

The second diagram gets closer to the value of this kind of site. Here, we see that the way in which each character experiences the events of the episode is quite different. This is crucial both to the plot and to the audience’s understanding and enjoyment of the episode. If, for instance, you wondered exactly how things tied together, then exploring this kind of site would allow you to piece together the parts of the puzzle. Perhaps on each character’s page, we would show their timeline, and how things happened to them. From the Doctor’s perspective, for instance, the event at the end of the episode is the first thing that happens to him – and the last from Sally’s point of view. Also, by showing these different timelines in the context of each other, we see the intricate way in which Steven Moffat (the writer) is able to weave the story together – giving the audience a greater appreciation of the story as a whole.

Obviously, Blink (so far) is an atypical episode of Doctor Who. By and large, the stories do not tend to concern themselves with the ‘timey-wimey’ stuff. However, over the course of a series, or indeed several series, characters, events etc re-appear – for instance the ‘Bad Wolf’ motif – the reason that the cliffhanger to ‘Turn Left’ works so well, is because it draws together elements of continuity established throughout several series. The audience gets maximum enjoyment out of such a moment because they are aware of the links and the context.

So what of the original series, whereby both ‘timey-wimey’ stuff and ‘story-arcs’ were at a minimum? Well, there are still instances of recurring themes, but overall, stories are self contained. That’s fine – they could be slotted into this kind of website just like everything else, because it essentially gives us a great pool of narrative to draw upon – if and when needed. Crucially, though, they represent a pool of ideas that future writers can draw upon if they wish. Continuity should not restrict the writing of future stories – the previous stories merely open out the fictional universe, creating more richness for authors. As such, when feeding the ‘classic’ stories into the website, the site becomes a form of ‘official’ wiki. Users can and should be encouraged to contribute, as a form of writing their own stories, but a distinction can be drawn between the events depicted on screen (it is, after all, and should not be forgotten, a television show..) and those where people ‘fill in the gaps’. The series itself has touched upon this, with the idea of certain events being ‘fixed points’ and others being ‘in flux’. As long as the narrative is not disrupted (i.e. breaks down so that it no longer makes sense to the audience) or becomes to insular (i.e. relying too heavily on continuity, so that new audiences are driven away), then continuity can enhance the fictional narrative universe as a whole.

Finally, a new diagram which, on a very basic level, tries to illustrate the idea that the website could be explored and presented through the model of, as quoted above, “a big ball of wibbly-wobbly…timey-wimey…stuff.” The diagram is quite obviously incomplete, but the idea is that the objects and the links between them are visualised, and the audience can then choose to look at a particular object, and see how it ties in to everything else – seeing both the object and its changing context and perspectives at the same time.

Phew. That’s enough for now. Till next time…