Unstructured Data: Creating the Analytical Environment

2 Day Seminar

Date/Time:	4-5 December 2008 Location: London	To Book ONLINE Now, Click Here!
Speaker:	Bill Inmon

It is estimated that 80% of data in the corporation is textual. There are emails, medical records, contracts, safety reports, patents, and a whole host of other forms of textual data. For years, managing textual data has meant placing documents in some form of ECM – Enterprise Content Management. But trying to do analysis on data found in ECM is a very different story than placing the documents there in the first place.

With DW 2.0 the idea arose that unstructured data is best placed in a data warehouse, where it can be analyzed along with other structured data found in the data warehouse.

This seminar is about the work that needs to be done in order to take textual data out of the confines of documents and integrate the textual data into a data warehouse. This is a very down to earth seminar/work shop. The first day is a lecture based on the background material needed to understand the architecture surrounding the placement of text in an analytical, data warehouse environment. The second day is a workshop that shows – step by step – how text is converted into a data base that can then be placed into a data warehouse.

There is a big difference between searching text and analyzing text. The seminar brings out these important distinctions.

The hardest part of transforming text into a data warehouse is the integration of the text. Anyone can read a text file and toss the text into a data base. Such an exercise is an exercise in futility. The resulting data base is one that cannot be usefully processed by a BI tool. In order to produce a meaningful result, the analyst must carefully transform the text. Some of the basic issues of transformation include:

• reading and understanding semi structured data
• applying external categories to text
• creating internal taxonomies of text
• standardizing dates for BI processing
• identifying patterned variables
• identifying named variables
• resolving homographs, and so forth.

There is a special emphasis on the management of corporate contracts and oil and gas pipeline and refinery safety data in this seminar.

Data analysts and business managers who need to know how to incorporate textual data into their decision making processes. In particular,

• the Data Architect who needs to know how to integrate textual data into a data warehouse
• the Business Analyst who wishes to form an analytic source of data based on textual data
• the Manager who recognizes that there is a lot of important data tied up in text and needs to know how to unlock that data into an analytical environment

• Recognize the difference between search processing and textual analytics
• How to create a data warehouse that contains textual data
• What is required to turn textual data into data that is fit for a data warehouse
• What internal and external taxonomies are and why they are important
• What synonym concatenation is and why it is important
• What homographic resolution is and why it is important
• What semi structured data is and how it has to be handled
• How to integrate the textual integrated data to a data warehouse data model
• What stop words, stems, and standardized data elements are and how to handle them
• How to create a foundation that can be analyzed by standard BI tools
• How to scale the integration process
• How to read and interpret semi structured data
• How to create proximity variables (and why proximity variables are so important)
• How to integrate textual data with visualization
• Understand the difference between textual discovery and textual analytics

Introduction

• Discovery
• Analysis

Search versus Analysis

• Transformation of text

Types of Unstructured Data

• Voice
• Image
• Text

Types of Textual Data

• Simple Unstructured Data
• Semi Structured Data
• Volumes of Data
• Textual versus Structured Data
• Scaling Volumes of Data

The Unstructured Data Base

• Iterative Development
• Imperfect Data

Integrating Simple Unstructured Data

• Stop Words
• Stems
• Synonyms
• Homographs
• Internal/External Taxonomies
• Alternate Spelling
• Multiple Languages
• Proximity Variables
• Date Standardization
• Text to Numeric Conversion
• Email (Screening)

Integrating Semi Structured Data

• Sub document Separation
• Looking at Hidden Characters
• Pattern Recognition
• Symbol Recognition
• Multiple Types of Indexing
• The Subject Oriented Index
• Index Trimming
• List processing

Linking Unstructured and Structured

• Dynamic links
• Static links
• By Name
• By variable
• By Communication Id
• By Business Id

Visualization

• SOMs
• Simple Unstructured SOM
• Semi Structured SOM
• The Discovery Process

Other Miscellaneous Topics

• Technology Infrastructure
• A Methodology for Unstructured Text

• This seminar is based on two books which are commercially available. The first book is Tapping into Unstructured Data, Prentice Hall, 2007. The second book is DW 2.0 – Architecture for the Next Generation of Data Warehousing, El Sevier Press, 2008. Both of these books are available and the attendee is encouraged to read these books before attending the seminar.
• In addition there is a collection of white papers to which the attendee will be directed.

Data Management Series

Data Modelling Fundamentals
23-24 September 2008, 16-17 February 2009, London

Data Modelling Masterclass
25-26 September 2008, 18-19 February 2009, London

DW 2.0: Architecture for the Next Generation of Data Warehousing
2-3 December 2008, London

Unstructured Data: Creating the Analytical Environment
4-5 December 2008, London

Master Data Management
15-16 September 2008, London

SERIES DISCOUNTS
Attend more than one course in this series and you will be entitled to the following discounts:

2nd course 10%
3rd Course 15%
4th Course 20%
5th Course 25%

Group Booking Discounts
If 5 delegates from the same organisation register at the same time for the same or various seminars, then the 5th delegate is free. We regret that this offer cannot be used in conjunction with the Series Discount.

£995 + VAT(174.13) = £1169.13

4-5 December 2008
Venue: myhotel Chelsea, 35 Ixworth Place, London, SW3 3QX
Tel: +44 (0)20 7225 7500
Fax: +44 (0)20 7225 7555
Website: http://www.myhotels.com/default.asp?section=10&page=1509

IRM UK in association with JP Events Ltd has arranged special discounted rates at all venues and at other hotels nearby the venue. Please visit the JP Events website for further information.
E-mail: enquiries@jpeventsltd.com Tel +44 (0)20 7428 9911 Fax +44 (0)20 7428 9966.

For an in-house presentation of this or any other IRM UK seminar, please e-mail jeanette.hall@irmuk.co.uk or call +44 (0)20 8866 8366.

© Copyright IRM UK Strategic IT Training, Bishops Walk House, High Street, Pinner HA5 5PJ UK, Tel +44 (0)20 8866 8366, Fax +44 (0)20 8866 7966, enquiries@irmuk.co.uk, www.irmuk.co.uk.