tfpl blog

Google's search results go social: the significance of SearchWiki

Last Thursday Google launched its SearchWiki which enables you to customise, contribute to, and comment on Google search results. If you are logged into your i-Google account you can:

• promote results you like and disappear results you don't
• add new results to the front page of a search for a particular term
• add comments against websites that other people logged on to i-google will be able to access when that same website appears in their search results.

It is a significant change.  Google have always shunned human influence over their algorithm driven search results, in order to ensure that they return the most authoritative sites for any given search term.  The price that Google have paid for this authority is a lack of currency and  human interest.

Robert Scoble posted here that  SearchWiki has been introduced because Google were alarmed at the short length of time that people spend on their search site.   People type in their search term, pick something from the first page and disappear.  This contrasts with the way people linger at sites such as Facebook and Delicious which have a social dimension.

SearchWiki is a bid to bring some colour and liveliness to Google's search results, but the tentative way that they have gone about it has drawn criticism from influencial writers in the technology sphere. Michael Arrington of Tech Crunch urged his readers to kill off SearchWiki by opting out of it. The ReadWriteWeb blog called it a 'train wreck' and criticised Google for putting the word 'wiki' into its title, without providing most of the functionality you expect from a wiki.

Google is the best search engine around when you are researching subjects that are unfamiliar to you. But if  you know a topic well then the first page of Google search results for that seach will be full of web pages that you are already familiar with.

For subjects that I want to keep up to date with (records management, SharePoint etc.) I prefer a visit to the following pages on the social bookmarking site Delicious. 

• http://delicious.com/tag/recordsmanagement
• http://delicious.com/tag/sharepoint
• http://delicious.com/tag/web2.0
• http://delicious.com/tag/dataprotection 

These pages display links to the webpages that have been recently tagged and bookmarked by Delicious users.  A brilliant blogpost may never garner enough inbound links to make up into the top few pages of Google, but one person bookmarking it in Delicious propels it, temporarily, to the top of the delicious page for the term they have tagged it with.

The social nature of Delicious provides an element of serendipity missing from the machine generated Google search results.   When I look at the resources tagged 'recordsmanagement' in Delicious I often recognise the names of some of the contributors, I can click through to the pages that collect together all the material that they have tagged, and in the process glean new links (and get to know a bit more about the range of interests of those individuals). 

Is SearchWiki an alternative to bookmarking your favourites in your browser or on Delicious? 

SearchWiki offering an alternative of sorts to social bookmarking sites such as Delicious.  I could choose to add a data protection website to the Google SearchWiki by

• copying the website url 
• logging on to my i-google account 
• searching on the term I want to remember the blogpost by (in this case 'data protection')
• scrolling to the bottom of the results page 
• clicking on 'Add a result'  and pasting in the website's url      

The next time I search Google for 'data protection' I would see the websites that I have added at the top, followed by the results that Google's algorithm has generated.

In practice Google is offering you less than Delicious:

• Delicious creates a page for you that brings together all the resources that you have bookmarked. This page is publicly accessible to allow others to benefit from your bookmarks
• Google does not offer you any home page to gather together the pages you have added, promoted or commmented on using the the SearchWiki.  Your sites are scattered under the Google search terms you have added them to.  The effects of your promotions and additions will not show in other people's search results.  Any comments that you have left against a particular site will, however, be accessible to other i-google users whenever that site appears in their results

How will SearchWiki cope with spam? 

Most web 2.0 applications start small, too small to be of interest to spammers. By the time they have grown large enough to attract the spammers attention they have established patterns of behaviour that mean that spam is easy to recognise and deal with. Google SocialWiki is different: the monetary and reputational value of Google search listings mean it is worth spamming from the off.

Micheal Arrington at Tech Crunch reported that a spammer had left a SearchWiki comment against the link to the Tech Crunch site, advertising their porn channel.     I did a search on Tech Crunch yesterday lunchtime and noticed that it had 282 comments saved against it, just a few days into the operation of SearchWiki. By comparison Barack Obama's home page had only received 42 comments at the same point in time. This implies to me that admirers of Tech Crunch have reacted to the blogpost by visiting SearchWiki and leaving positive comments to drown out the spam: a similar behaviour to that which protects Wikipedia pages against vandalism.

SearchWiki does not provide users with the standard tools that most wikis provide to deal with vandalism. If you see spam on a Wikipeadia page you can get rid of the spam and revert to a previous version of the page. With SearchWiki you cannot touch the spam comment. 

The big risk for Google is that people will leave offensive or spam comments against the websites of companies that are giving Google their revenue stream by paying to advertise with them. If campaigners left hundreds of comments criticising a company for, say, its environmental record, would Google continue to leave that SearchWiki open? Would that company continue to pay to advertise with Google? 

TechCrunch have a loyal readership who have defended them: what about unloved websites? Several bloggers have asked whether Google got any method for defending them against spam comments. 

The user experience of SearchWiki

If you want to see the SearchWiki comments relating to the TechCrunch site then follow these steps: 

• Log on to your i-google account
• do a google search for TechCrunch,
• scroll down to the bottom of the page and click on 'all notes for this SearchWiki' 
• go back up to the listing for the TechCrunch site and click on the thought bubble beneath the link 

You can see from the number of steps I've described here that Google has not tried to make SearchWiki user freindly. This is an experiment - Google do not want to make the comments too prominent whilst they are unsure of what the comments will actually be like. 

If Google like the way SearchWiki is going I would expect them (or third party developers) to provide browser buttons giving shortcuts for leaving SearchWiki comments and for promoting, demoting or adding a site to a particular search result.

The opportunities and risks for Google of going social

The big opportunity for Google is that the comments feature on the Search Wiki could become a forum for the rest of the world to give feedback on, and to debate, every extant website. If you think a company's website is too superficial, rather than send a comment to their website you might leave a comment against the website on the SearchWiki, to be viewed by anyone logged on to i-google.  

If the SearchWiki comments end up being informative they will provide an incentive for people to log into their i-Google account before searching, which will increase the likelihood of them taking up the other services that Google offers. 

If SearchWiki takes off I expect Google to use the data from Search Wiki promotions and additions as an input into a search results feature that displays 'what is hot' : websites or blogposts that are rapidly rising through the rankings and being frequently promoted or added.

The risk for Google is that they fail to please anyone,  they may end up:

• offending the web community and blogsphere by not providing enough social features
• offending companies that pay for their advertising by exposing their websites to criticism and spam when they appear in search results.  

Can Google walk the tightrope?  Will the sheer size of the audience for their search results mean that people will leave comments and companies will pay for adverts regardless of their opinion of SearchWiki? 

Watch this space!

Is blogging good for your career?

Joseph Jaffe has written this useful blogpost this week outlining the reasons why, in a credit crunch, blogging can be useful for your career.

I agree with his assertions that blogging regularly about your profession:

• forces you to stay up to date with your field
• helps build your network
• maintains your visibility in your profession

If you don't want to keep a blog of your own then commenting on other people's blogs can bring similar benefits.

If you are informative and keep a positive tone the risks of upsetting people should be small, unless they object to the very fact that you are blogging at all.

The key thing is to be sensible:

• Don't say anything about an individual that you wouldn't say to their face
• don't say anything about your organisation that you wouldn't say to your HR officer in a loud voice on a crowded bus.

Assume that if you mention an individual or an organisation by name that they will find and read your post. Your organisation may not be interested in your blog but it is interested in its own name and its own brand. Most organisations and many individuals have searches set up that trawl the intranet for any recent mentions of their name.

These points apply not just to blogging but also to participation in any form of online forum or community. The Berkman Center for Internet and Society at Harvard have posted up to their Digital Natives blog four tips on keeping private those opinions or pieces of information that you don't want to be widely available online.

They point out that it is in the interests of social networking sites like Facebook for the information you contribute to their sites to be as wideley accessible as possible. For this reason their default settings tend to favour openness over privacy. A good example of this was the case of the 18 year old Crystal Palace footballer footballer who was having a trial with rivals Fulham. He thought he was just telling his 198 Facebook friends about the trial but his message was actually accessible to his fellow 2.7 million members of the Facebook London network.

Blogging is now one of those generic communication skills, like presentation skills, and report-writing. You will be able to use it in different ways during your career. As an information professional you may find yourself in a job where you want to write an internal blog. This will enable you to give your colleagues the benefit of your insights into the information challenges your organisation is facing and the projects that you are working on.

More and more and more organisations are providing their employees wih the means to blog internally. Even if your organisation does not provide a specific blogging application to blog within the firewall, the latest version of SharePoint provides blogging functionality (albeit rather basic) as a standard part of each individual's SharePoint mysite.

Can libraries prosper in a Google dominated information world?

The 'Google generation' are defined as those people born in or after 1993, who cannot remember life before the world wide web. They will be hitting universities in three years time.

In 2007 JISC and the British Library funded the Google Generation research project to identify the implications for university libraries of the imminent arrival of a generation who for their whole life have needed to do nothing more to retreive information than to type some words into the blank Google search box.

The report came out with a stark message for university libraries. Never mind the Google generation - the current generation of students and researchers have just as strong a preference for using Google and the web in place of using the catalague and the library. This Thursday I attended a seminar run by CILIP's Information Services Group entitled The digital consumer: our customers of the future . The seminar explored the report's finding that:

Libraries could be "swept aside by history" if they continue to fall behind the internet in addressing the changing demands of researchers and students

The findings of the Google Generation report

The first speaker was one of the co-authors of the report: Dr Ian Rowlands. He explained how the report had found no evidence that the Google generation is qualitatively different in its information habits than those in older age groups.  Only a minority of the Google generation live up to the stereotype that they are constantly on-line and are highly tech savvy. Most of the rest take a pragmatic approach to web 2 (using it when it serves a purpose). Another group are 'digital dissidents' tending to dislike or shun it.

His description of the google generation tallies with my experiences of colleagues of all ages in most of the organisations I work with:

• a minority love web 2 and will be blogging or twittering or on Facebook every day
• the majority will use it for specific purposes (e.g. accessing a blog about a topic they are interested in)
• another minority won't touch web 2 with a bargepole.

The report found that the main difference with the google generation and those older than them is that because of their stage of life they have more time on their hands to use web 2 and the internet, and they have less pre-conceived opinions about information sources.

The good news that the next generation of library customer is not an alien species brought little comfort to the assembled librarians because Ian rang out some sharp warnings:

• The Google generation is not good at framing precise searches, and does not use advanced search facilities. But current students, the middle aged and the old are exactly the same! This is a big worry for libraries because these are precisely the skills students need in order to derive the most benefit from library resources
• Students prefer convenience to quality when it comes to information. Ian often asks his students: what is more important to you, ease of access to information or the quality of the information content? They always answer ease of access. This is worrying for libraries because their unique selling proposition over the web is that information in libraries is of a higher quality.
• Students are not attracted by good information architecture. Ian was asked by a publishing company to run a workshop with PhD students to explore how they wanted the publishers e-resources site to be structured. They said they did not want clouds of meta-tags or faceted classification, they just wanted a simple search box like Google.

The Google generation are no different from current students and researchers and the rest of population. The mismatch of running information architecture centric libraries for information architecture phobic users is not a problem that will arrive further down the track, it is a problem for all libraries now, whatever audience they serve.

The challenges for digital libraries in the web 2 world
In theory digital libraries are a solution to the problem of providing the population of a university with convenient access to high quality information.

In practice Ian said that digital libraries in universities were threatened by a lack of usage and by the risk that publishers will provide researchers with direct access to e-resources - bypassing the University library.

Ian ascribed the cause of the under-usage to the fact that students and researchers have difficulty in forming a mental map of the scope, structure and contents of a digital library. They don't distinguish the contents of the digital library from the big pile of information that is the world wide web.

Ian urged librarians to do more to help people understand what is in the digital library, what makes it different and special. He was disappopinted with the home page of the UCL digital library which simply offers users a choice of 'e-books' 'e-journals' and 'other repository materials'. This distinction based on the format of information makes sense in terms of the way that librarians process their stock, but it doesn't relate to the reasons why students and researchers consult the digital library and it doesn't convey any sense of the rich array of resources available within the digital library.

Ian believes that initiatives of publishers threaten to encroach on the space occupied by digital libraries. Publishers can take on libraries because of the content they possess. They have also put more time and money into understanding the information habits of the university community than librarians have. Elsevier are planning an e-information resource that aims to meet the information needs of researchers whilst they are doing their research, in the hope that the researchers will publish their papers with them at the end of the research.

Teaching information literacy in the web 2 age
Peter Godwin is Academic liaison librarian at the University of Bedfordshire. He writes a smashing blog on information literacy, and he spoke to us about how he goes about giving students at the university the information skills they need to prosper in the web 2 age.

He criticised the bulk of information literacy training given by university librarians for over-concentrating on search. The reason for the focus on search was that the librarians wanted students to use their library databases. Students don't need to think like librarians (that won't make them rich! quipped Peter) they need to know how to collaborate, synthesize and adapt.

Peter talked us through Sconul's seven pillars of information literacy. To Peter the most important information literacy skill for students to acquire is the ability to evaluate and compare sources of information.

Peter says that web 2 rewards people who are willing to experiment and try things (it is not as well-suited to perfectionists). So his teaching method involves encouraging students to engage with these tools. He doesn't just teach them how wikipedia works and what its strengthts and weaknesses are, he encourages them to go to a topic they know a lot about and add to the wikipedia entry. He doesn't just teach them about the nature of blogs and how to follow them with an RSS reader, he gets them to consider whether writing their own blog would help them in their studies.

The University of Bedfordshire have set up a 'Course Feed' feature, that allows students them to view within Facebook information about their course drawn from the Universities Virtual Learning Environment system.

Juanita Foster-Jones is a librarian at the Open University. She descibed how she has collaborated with academic colleagues to put together two information literacy courses that the OU offers to members of the public. The first is 'Beyond Google: working with information online' an undergraduate course. A second course is being launched this December. Putting the course together enhanced the status of the library within the University and got them working with academics as well as for them.

The use of web 2 and the web by the British Library (BL)
Clive Izard is head of creative services at the BL. His aim is to use web 2 and the web to create connections between the experience of people visiting the Library's reading rooms, its exhibitions and its website.

He spoke in particular about two projects:

• Turning pages 2 is aimed at overcoming the difficulty of exhibiting rare books. You have to put a glass case between the visitor and the book, you can't let them touch it and you can only show 2 pages at a time. BL wanted a software programme capable of taking images of individual pages and binding them into a virtual book. They wanted the software to enable users to interact with the book: to rotate it; turn the pages; magnify text and pictures; click for notes, translations and further links. To achieve this BL partnered with Microsoft. Both parties benefited from the partnership: Microsoft got a flagship project that they used to launch Vista. BL got to retain the intellectual property in software that they can use again and again, and that they have provided to other libraries to use with their collections. Clive showed us the Turning Pages rendition of a William Blake Notebook. The stunning look and feel of the virtual book combined with the beauty of Blake's artwork took my breath away.


• Visitors to the Taking Liberties exhibition are given an anonymous citizen number and invited to give their views in response to questions at various points around the exhibition. The questions hit on controversial topics such as 42 day detention. Visitors to the website are given the same questions. Visitors to the exhibition are fed information on how people on the website are voting on the questions and vica versa. Both sets of visitors are given an individual reading of where their views stand on a continuum between control and freedom. You can start the questions at home and finish them on the web or do it the other way round.

BL are using web 2 technologies to make their own web site more participative. But they also want to enable people to access BL collections without having to come to the BL web site. so they contribute photos to flickr, videos to You Tube, and have contributed to Facebook.

Clive says that the main driver behind software and technology change now is the consumer market not the organisational market (think of the iPhone). Technology change is so rapid that things look old hat after 18 months. The BL can't hope to develop new technology in-house, it needs to partner with companies that specialise in this field. Luckily because the BL has rich content that is not currently available on the net the big boys like Google and Microsoft are willing to work with the BL on favourable terms.

Post seminar thoughts
Cilip's HQ is nestled in the heart of Bloomsbury, tucked in between the tech shops of Tottenham Court Road and the teach shops of UCL and the University of London. After the conference I popped into the The Building Centre on Store Street: an exhibition area and bookshop that promotes new architecture.  I saw an architectural magazine with the headline  'if you want to change the world, don't build anything'. The headline referred to an architecture firm who only build buildings as a last resort. When a client issues a brief they use their knowledge of how to manipulate and use space to explore and propose options which don't involve new construction.

I was struck by the parallels with the debates I had just heard: speaker after speaker had explored ways of enabling people to benefit from the skills and knowledge of librarians without requiring them to visit a physical or digital library.

James Lappin

SharePoint and the management of data

There are some functions of enterprise information systems that Web 2.0 and cloud computing services can easily replicate. They are perfectly capable of giving knowledge workers the applications they need to:

• store and share documents
• communicate with each other through e-mail and messages
• collaborate with news, calendars, tasks, photos,blogs and wikis

The fact that these applications are offered either free of charge or cheaply by web 2.0 providers is a threat to vendors of collaborative systems and of document/records management systems. Organisations may find themselves powerless to stop staff migrating away to web provided applications either as teams or as individuals.

However there are other types of enterprise information systems that it will be difficult for Web 2.0 services to replicate, namely:

• systems that exist to hold, crunch and analyse data
• systems used to re-engineer a business process or processes with workflow

These systems tend to require too much customisation and too much integration into other organisational systems for web 2.0 services to replicate.

In its early manifestations SharePoint was primarily a collaborative system. If it had stayed that way it would now find itself very vulnerable to competition from web-based rivals such as Zoho and Google Apps. The game-changer with MOSS 2007 is the Business Data Catalogue (BDC).

Microsoft want your organisation to use the Business Data Catalogue:

• to bring data in from your various databases (your finance database, project management database, customer database etc.) for consultation and use within SharePoint
• to provide your users with an enterprise search facility: allowing them to search across the various databases of your organisation
• to use data imported from other systems to control and populate metadata fields in SharePoint. In effect using your existing databases as controlled vocabularies in SharePoint.

Microsoft's sales pitch for the Business Data Catalogue is that it enables you to surface data from business databases in SharePoint without any coding. Simply describe the information architecture of each of your databases in XML and then let the BDC search over, and use data from them. The best descriptions of how the BDC functions that I have read on the web are by DotNet Dreamer and by Martin Kearn. Martin Kearn describes the BDC as the most powerful feature of MOSS 2007.

The business data catalogue is a masterstroke in Microsoft's tenacious battle to remain top dog in in providing the information management platform to organisations. Microsoft are inviting organisation's to use the BDC to bind their SharePoint implementation in with their core databases and line of business systems, a role which at this point in time none of its web based rivals in the collaborative space could replicate.

The Business Data Catalogue threatens to burst apart the long standing distinction between systems dealing with structured data (databases) and sytems dealing with unstructured data (document management systems and collaborative systems). It will have implications for us information professionals - how many of us have skills and experience that span the two spheres of structured and unstructured data?

The best implementations of SharePoint that I have seen have been the ones that used organisational data creatively within SharePoint.

At the 3rd TFPL SharePoint Conference Stephen Wilder described how The Leprosy Mission International (TLMI) had used the BDC to synchronise SharePoint with their project management database.

• Whenever anyone in TLMI saves a project document to a SharePoint team site they identify which project the document relates to from the list that the BDC draws from the TLMI project management database.

• Other SharePoint metadata fields for the document are populated from information about that project held on the project management database

• Within SharePoint TLMI have built a portal site containing a project directory. The directory lists each project, and links to all the documents saved against that project, in whatever SharePoint team site they were saved in. This in effect creates a virtual project file. Individuals get to work in their local team space, but a corporate view is provided of the work on each project through the project directory.

James Lappin

The National Archives position on EDRM

In my last blogpost I wrote about the way that the UK National Archives has modified its stance on EDRM over time.  

Natalie Ceeney, Chief Executive of the National Archives, has left a detailed comment to the post, in which she explains the current position of the National Archives on EDRM.

The clarification is very useful indeed,  here is a full copy of Natalie's comment:

Given this post, it might be helpful if I explained The National Archives' position on EDRM?

It seems clear to me that, despite massive spend and huge energy on EDRM solutions, we haven't solved the issue of making sure we keep electronic records effectively. Why? Because I don't think we've been asking the right questions or have approached the challenges of e-records in the right way.

We believe that the first question to ask needs to be 'what do we need to keep?' and then, and only then 'so how do we keep it?'. If organisations did this systematically, most would probably find that some bits of their organisation create far more critical records (e.g. Procurement: legal contracts, CEO's Office: board minutes, HR: personnel records) than others. And they'd also identify a number of important records needing to be kept which were not on people's personal systems - e.g. discussions happening on intranets or wikis, or blog sites, or web announcements. And they'd probably also find huge swathes of their organisation which never create or use any of these records beyond their initial reading. We have to start here if we know what problem we're trying to solve.

Only when we've done this should we consider 'capture'. The problem with EDRM implementations is we've seen them as 'the solution' whereas, in reality, they are part of a jigsaw of solutions depending on what you're trying to keep. The solution to capturing the wikis and blogs might be web harvesting, allowing the discussion to continue freely. The solution to keeping records of contracts and associated negotiations might be to set up an automatic feed from the contracts team's shared drive into a secure repository. And for only some of the key records, and for some of the organisation, the answer might be an EDRM system. And even there, the 'answer' might be to use EDRM as a back office infrastructure component and not a front end interface, harvesting clearly identified information from email boxes or shared drives. And we may not need to pay for licenses for everyone if not everyone needs to use it.

And there is another key component here - our users. Records are rarely created by anyone other than as a by-product of their job. And the main beneficiary of records is usually the organisation, or others in the organisation, and not the creator. The problem with many EDRM implementations is that we've made the user do more work. And so that user has a choice - work more hours, or skip the filing. Unfortunately, most of us know that the latter tendency predominates. And this is a potential disaster for organisations whose reputation and financial stability depend upon good record keeping. We need to make record capture easy, and ideally invisible to the creator, whilst focusing the creator of records on the need to do their job, and record decisions, well. This will make their lives easier, and really help organisations in terms of risk and governance.

So, in summary, The National Archives does support 'EDRM'. But only as a tool for capture, not as 'the solution to e-records'. The priority for our community is to make sure that the key information created is kept for appropriate periods - for the sake of the long term of the organisations we support. Starting by asking about 'what needs to be kept' rather than about EDRM might take us far further forward to achieving this goal.

And finally, in response to comments above [I had mentioned that Natalie was on the interview panel that appointed Roger Smethurst, who has expressed scepticism about EDRM, as Head of Knowledge and Information Management of the Cabinet Officer], I can assure anyone reading this that when interviewing people for senior roles, I am looking for a broad range of information management as well as influencing and wider engagement skills, and do not base my assessment on their position on EDRM technology!!