NewsScape

Introduction

NewsScape presently exists in two forms – NewsScape Version 1 – a vanilla html presentation at www.newscape.com that delivers to just about any html browser and device.

NewsScape 2 – a development project – lives at dev.newscape.com, and this parades a number of advanced ideas and features but requires IE version 6

Overview

NewsScape is the most effective way of summarising and categorising the vast journalistic output of the internet

NewsScape is more than this: it is a way of presenting, in a readable format, a summary of any published material on the Internet, or an intranet. It can be selective too, selecting just the type of information that the user requires.

In its simplest form, NewsScape cycles through a list of sources and detects any new additional material on those sources. It formats the material into a title, summary and text. It then indexes, analyses and displays this material. With News Sites, it can list the various categories of news in a number of ways, and do complex searches for news items. It can sort stories into categories according to simple or complex criteria, or alert users to the presence of newly published material from a particular source, or with particular content.

NewsScape can automate a great deal of the drudgery of checking news sources, whether for journalists doing research, company intelligence, checking brand integrity, or just simply for routine reports on what is being said about particular organisations

A strength of NewsScape is its powerful searches. It can search through 4.5 million news stories for particular words, or strings in under a second. Even complex searches take around four seconds on a small twin-pentium machine. NewsScape is able to do fairly complex processing on its stored materials. It can, for example spot major stories published by several sources from the underlying 'noise' by spotting changes in the use of words and phrases (e.g. Soham or Bali), and categorise them as 'featured' stories.

NewsScape has been used to monitor stories on a particular subject (e.g. Commerzbank), or to research the history of particular ongoing news stories. Unlike other news stories it has a three-year archive which allows in-depth historical research

NewsScape is able to be catholic in the areas of the web that it inspects. It can get behind bland press-releases. NewsScape, for example, scans product recall sites, notices of creditors meetings, court reports, trading-standards judgements, newsgroups, and various government sites. Because it is so economic to use, it can be set to inspect primary sources of news, press releases, occasional magazines or political material.

NewsScape is an information aggregator. Although it has been developed primarily to index and categorise news stories, it is of use of anyone who must make sense of a huge mass of information, most of which is irrelevant, and who must be able to browse and search through it. NewsScape has evolved to be more and more efficient, as its database becomes more and more enormous. Its searching algorithm is complex but startlingly fast. It is therefore an excellent way of presenting the information contrained in intranets.

Who needs NewsScape

·         Anyone who has blown a business opportunity because of missing that morning's industry announcement

·         anyone who finds it impossible to keep track of what your competitors are doing, throughout the day, every day

·         anyone who wants to know what's being said about their company, competitors, clients and partners - in time to act on it?

The sheer volume of online information doubles every year. But it's the right information that continually eludes professionals: 79% turn to the Internet first for information, and spend more than 20% of their time locating the information they need. (source: Outsell)

Today, success is almost impossible without access to information that is:

·         Targeted to the users' unique business needs

·         From the most influential online sources

·         Fast enough to impact decisions

Professionals are continuously missing vital information they need simply because they don’t know how or where to find it in a world of endless information. Then, if they do manage to find what they need, it’s rarely in time to put it to good use.

By turning overwhelming amounts of information into actionable knowledge, NewsScape aims to enhance global intelligence and competitiveness by managing real-time data streams and delivering them immediately and intelligently.

NewsScape’s technology locates, filters and consolidates a breadth of real-time information from countless content sources – such as Internet News-sites, broadcast television and radio, news wires, web pages, message boards and more – and delivers only what is relevant.

By discovering the most useful information in real-time, NewsScape gives you the entire universe of information transformed into useful knowledge. NewsScape is the perfect tool for anyone who needs to receive relevant information as it is generated. It is especially useful in such industries as financial services, public relations, news and government organizations or any other knowledge-driven industries. With NewsScape, not only do you have all the breaking news, but you also have it organized according to your personal preferences in one centralized location.

As news and information break, they immediately “find” the appropriate recipients automatically. Users determine the flow of information to them by creating a network of queries (in the form of keywords and Boolean expressions) that encompass the subjects that are valuable to them in their daily operations. These queries put them on the edge of real-time information as it emerges across all global information sources. They do this by alerting the user to their search results in real-time, thus keeping them on the edge of information that’s relevant to them. The aggregation of the query results becomes a snapshot of the most up-to-date information available. Alerts are created and categorized into folders by subject, company name, etc. NewsScape cuts through information overload to provide users with only the information they need, when they need it most. Managing countless data streams and personal criteria, it gives users the ability to monitor only the information that pertains to their individual needs, and set alerts to notify them of real-time information like breaking news and financial updates.

NewsScape delivers the breadth of external information coming from such sources as television broadcasts, news articles, website updates, and more. NewsScape offers several choices as to the types of content packages available so that you receive the most relevant information delivered to you instantly in an organized, personalized interface.

With the exponential growth of live content streaming through the ether and growing demand for personally relevant information, individuals in all walks of life need new solutions enabling smart and active information navigation and retrieval. NewsScape can coordinate and deliver the best of these solutions, enabling significant penetration and expansion of several major markets.

Initial applications for this type of product are traditionally aimed at the $21.5 billion financial services segment of this market. However, that market is way oversubscribed and the incumbents are suffering. But those who once had a $1000/month Reuters terminal on their desktop might not be suffering from information withdrawal, and find that newsscape actually fulfils most of their real requirements.

NewsScape’s solutions are ideally suited for this industry, where real-time access to information is critical.

 

How NewsScape works

NewsScape is a combination of database, DataGatherers and Website. The current production NewsScape database is on an NT machine with SQL Server. It is a twin pentium machine with three Gigabytes of memory, which is little more than an averge desktop specification, these days. The DataGrabbers host a process that runs on rack Pentium 3 NT boxes with access to the Internet. DataGrabbers trawl through a list of sites, at scheduled times according to the volatility and importance of the site. NewsScape presently gathers over 12000 stories a day from over 3000 sources. There is no practical limit.

The entire logic of categorising, indexing and analysing data is written in Transact SQL, and is highly optimised. It has been tested at a sustained rate of 2000 stories per hour. The only requirement for remote maintenance is to maintain a table of sources. NewsScape has run unattended for several months

NewsScape collects the content of a block of information ('story') and indexes it. It is able to scan it over to check for key phrases, search for proximity of words and alert individuals. It can 'sniff out' the subject matter of a story from the presence of 'diagnostic words', and it can display information or news on a particular subject from a large number of sites as it comes in.

NewsScape and its' history

NewsScape was originally designed as a news-aggregator with a historical dimension, optimised for reliability and low production overhead. It uses a number of table-driven 'DataGrabbers' that trawl through the sources at scheduled times, and presents them in a browser-based summary or 'news-Ticker'.

NewsScape has been running for three years as a system for providing an overview of ephemeral public information, . It has been gathering an archive of news items, and allows this archive to be searched for items of interest. Although it is a public site, it has never been exploited as a revenue source. It has been primarily serving as a feed for news stories through a multicast feed to the EnfoCast application.

Unlike the Internet-based news services, NewsScape is not restricted to the primary news services. It also gives a historical dimension over the past three years. Currently NewsScape gathers news, stock-quotes and broadcast-media listings. It is now starting to collect product-recalls, reviews and other trader-based information. NewsScape also contains Court reports, including details of employees who take employers to industrial tribunals and all libel, deformation and slander cases.

 

The NewsScape architecture

NewsScape consists of a database, a collection of newsgathering processes, a website, a queuing system, a back-end reporting system and a messaging system

NewsScape is best seen as a number of inter-relating processes. Currently, many of these are co-hosted on the one server, but this need not be the case. As the system grows, it can be increasingly distributed. Already, there is a customized replication system in place. This can be extended so that more complex tasks can be offloaded from the main servers.

The DataGrabbers

These are processes written in a scripting language (Perl). The are programmed to gather stories from a specified source. Datagrabbers can be sited anywhere on the network. When one of them finds a story, it is checked for a match with existing stories. If it is new, it is indexed and saved, along with its' keywords. The user of NewsScape can then see the news, reports, or commentary as it is found. There is currently only one DataGrabber box, but this can be expanded up to any number. DataGrabbers are processes written in a scripting language (PERL) that are programmed to gather stories or information from a specified source.

Information on how to retrieve the stories from the source are contained in a source table on the database, editable by the NewsScape editorial team. The DataGrabber determines all the parameters for the source via an ODBC link. It writes its' progress to a table

The main NewsScape Website

(www.newscape.com) has now been running in its present form for three years, and was preceded with a prototype design. It has not been fundamentally altered since. It currently displays lists of stories including the title, summary, URL, and the time it was added. These lists are prepared according to the category (e.g. World, business, Technology). There are also lists, in time order, for the breaking news (i.e. the latest stories that have been added) and there is a list of 'featured stories' originally intended to be selected 'by hand', but now selected automatically.

 

How NewsScape is developing

NewsScape harnesses the world’s editorial content – from broadcast television to newswires to webchat – as it is being published, enabling subscribers to tap into the totality of public perception immediately and extract personally-relevant information. NewsScape’s solution is based upon three core elements:

·         Real-time access to the world’s most compelling content, including TV, radio, newsfeed, streaming media, web, bulletin board/chat, and intra-company content,

·         A proprietary real-time filtering platform capable of filtering tens of thousands of these feeds and intelligently extracting relevant content based on user profiles, and

·         A globally distributed publish-and-subscribe network infrastructure capable of securely connecting users with relevant content via multiple devices, all with end-to- end latencies of a few seconds.

The result is a new era in communications, an era where content immediately and intelligently finds interested individuals anywhere in the world. NewsScape’s revolutionary solution embodies the evolution to the next-generation Internet – from archival search and retrieve to real-time publish-and-subscribe, from computing to communications, from PC-dependence to device-independence.

increased user-customisation

Users can be alerted by email or SMS (or any other practical means) when certain key words occur at a good relevancy score. NewsScape can issue a daily/weekly email newsletter. NewsScape is configurable as a corporate Intranet site, with the identity and style of the intranet. Customisation options for users allowing them to change graphics, backgrounds, customise News-feeds and ticker etc

catering for workgroups and communities

NewsScape has rudimentary tools to facilitate workgroups, allowing them to create their own new channels, pass comment on news stories and rate them with the top rated stories going into their own column. NewsScape will soon be able to utilise its user information database to match users with similar profiles - and suggest further content that they might have an interest in.

There will soon be a mechanism for allowing users to apply comments to news stories that are visible only to other group members.

B2B Information Content

Outsell Inc., a research firm specializing in information services, once sized the B2B Information Content (IC) market at over $50 billion. This market includes publishers, aggregators and distributors of information content.

·         Financial Services. Management plans to focus its initial applications on the $21.5 billion financial services segment of this market. NewsScape’s solutions are ideally suited for this industry, where real-time access to relevant information is critical.

·         Market Research. NewsScape’s real-time platform and infrastructure will enable a new genre of media metrics measurement for this $15 billion market. The two-way nature of NewsScape’s solution will enable detailed usage measurement and the development of a new class of “Nielsen-type” metrics.

·         Reference & Aggregation. This $8.3 billion segment is currently dominated by purveyors of text-based archival information such as Lexis-Nexis. NewsScape’s ability to deliver rich media in real-time promises to add significant value to this market segment.

·         Public Relations/Crisis Management. NewsScape’s ability to comprehensively capture the world’s perception of a given topic will add considerable value to firms engaged in public/investor relations and crisis management, and will lead to considerable penetration of this $3 billion market.

News Production

NewsScape’s familial relationship with EnfoCast provides access to the world’s news organisations, and we have to properly leverage this opportunity. We realise that a formal business proposition is a precursor to meaningful discussions – but our ability to instantly deliver relevant content to news desks around the world promises to give NewsScape a foothold in this $30 billion market.

Convergence

NewsScape will deliver crucial applications for emerging markets spawned by convergence.

·         Wireless. Goldman Sachs recently quantified significant opportunities created by the high-bandwidth Mobile Internet, forecasting U.S. 2005 revenues of $4.4 billion for mobile commerce and $3.0 billion for advertising. We believe that NewsScape, with its ability to deliver highly-relevant rich media content in real-time, will be a killer app driving adoption of video-enabled mobile devices. Thus, NewsScape stands to capture a significant share of these new revenue streams.

·         Smart TV. Forecasts for the global  Smart TV market to reach $45 billion by 2005, with electronic program guides in >100 million households and personal video recorders in 70 million households by then. By analysing metadata (e.g., subtitles) of television content as opposed to merely scanning program information like existing solutions, NewsScape's proprietary technology represents the evolution of electronic program guides to electronic content guides, and the company stands to realize considerable licensing fees from existing providers (similar to Gemstar's approach).

·         Enterprise Solutions NewsScape’s solutions can empower organizations of any size and scope with a critical knowledge management tool that leverages internal assets, from employee communications (e.g., e-mail) to company publications to customer information.

·         eCRM. According to IDC, the “collaborative service and support technologies” segment of the eCRM market is projected to grow from $4.3 billion in 1998 to nearly $12.0 billion by 2003.

·         Corporate Portals. The Aberdeen group anticipates revenues from corporate portal products to top $1.3 billion by 2003. Additionally, knowledge management services are forecast by IDC to top $4 billion by 2003.

·         Other Markets. NewsScape has identified and will continue to explore several other potentially attractive markets, including sports, gaming, government, legal, education, and research.

NewsScape will build upon its first-mover advantage by partnering with the most highly relevant and respected content sources from around the globe. Users can simultaneously access tens of thousand of content streams across all forms of media in real-time. The result: users can tap into the totality of perception of any given topic at any given point in time.

·         TV/Video. NewsScape is currently pursuing licensing agreements with CNN, CNBC, Bloomberg, BBC, JAGfn, WebFN, On24, and others.

·         Radio/Audio. NewsScape is currently working with broadcast partners as a result of our relationship with EnfoCast.

·         Web Page Updates. NewsScape is constantly scanning over 4,000 premier websites for breaking news and updates through an agreement with Moreover.

·         Internet Community. NewsScape is currently monitoring IRC chat rooms and Usenet newsgroups, and is pursuing licensing agreements with Raging Bull, Silicon Investor and others.

·         Newsfeeds. NewsScape currently access over 15,000 articles a day over 50 wires through an agreement with Comtex, including AP, Business Wire, UPI, Knight Ridder, and others.

·         Data Feeds. NewsScape is evaluating licensing of real-time market data and other real-time feeds from the usual suspects. It plans to develop it’s own index technology specific target markets.

·         Instant Access. Users will be able to view the live video of their latest alert, or view the live broadcast of any content stream they choose. Additionally, the most current piece of relevant content from across a given user’s profile is constantly displayed.

·         On-demand Access. NewsScape is intuitive.

·         Archive: 3 years of news stories (4 million items) from around the world are already available (text only)

 

The NewsScape Team

An experienced and respected team of technology and media veterans drives NewsScape development.

NewsScape’s business operations are based in Chelmsford UK, and its R & D operations are based in Suffolk. There is an experienced Editorial team that monitors new News-sources on the internet for inclusion in the source-list

 

Commercially

NewsScape has grown alongside other products in the enformatica stable. It has continued to thrive and evolve as one of the development teams favourite in-house research tools.

The time is now right to separate NewsScape as commercial opportunity in its own right, since its users agree that it is superior in all respects to the competiton from the likes of MoreOver and NewsNow.