Wednesday, 26 June 2013

Pattern Matching with Oracle Database 12c

Now that Oracle Database 12c has launched you will see that we have updated the SQL Analytics home page on OTN, which is here: http://www.oracle.com/technetwork/middleware/bi-foundation/index-092760.html.
 
One of the new things listed on our OTN home page is a very exciting feature headlined "pattern matching". Those of you who are using Warehouse Builder will remember the regular expression syntax that was added into the Paris 10g release and you are probably thinking that "hey, I thought Oracle Database already did pattern matching". Of course, you are quite correct. Prior to Oracle Database 12c you could search for patterns within a specific row of data using all sorts of techniques. However, this new pattern matching feature is something is really quite special.
 
The Oracle Database 12c adds SQL pattern matching capabilities brings the simplicity and efficiency of the most common data analysis language to the process of identifying patterns within a data set. It provides a completely new native SQL syntax that has adopted the regular expression capabilities of Perl by implementing a core set of rules to define patterns in sequences (streams of rows). This new inter-row pattern search capability complements the already existing capabilities of regular expressions that match patterns within character strings of a single record. The 12c MATCH_RECOGNIZE feature allows the definition of patterns, in terms of characters or sets of characters, and provides the ability to search for those patterns across row boundaries. It offers significant gains in term of performance, maintainability and scalability compared to the continued use of bespoke code embedded within applications and executed either on the client side or within the middle-tier application server. 

For business users and data scientists the key advantage of in-database pattern matching is that it allows them to incorporate pattern enrichment into their existing analytical workflows without having to resort to complex externally coded routines that increases network traffic and increase data latency. This whitepaper will explore this new SQL functionality and explain how different industries can leverage the full richness of Oracle’s new SQL pattern matching capabilities.

Moving forward developers really think seriously about switching their pattern matching logic to the new SQL-based pattern matching features offered by Oracle Database 12c so they can benefit from simplified code, more efficient processing and improved performance.  
 
I have posted some articles and presentations on the updated SQL Analytics page on OTN.  When you visit the updated OTN page look for the section marked Technical Information . Here you will see links to a whitepaper and presentation from OpenWorld 2012 that will help you to understand this great new feature:
 
red-arrow-box SQL Pattern Matching: Patterns Everywhere - Find then fast! 
red-arrow-box Analyze This! Analytical Power in SQL, More Than You Ever Dreamed Of (OOW 2012)
 
 In the section marked Training you see a list of three podcasts:
 
red-arrow-box Podcasts: SQL Pattern Matching
Part 1 - Introduction to pattern matching
Part 2 - Concepts, syntax and demonstration
Part 3 - Uses Cases
 
 
I will posting three articles on this blog over the next week; one for each podcast (please note that the links on OTN for each podcast will come through to these upcoming blog posts so at the moment the OTN links are not actually working). These will introduce you to the new Database 12c pattern matching feature, explain how to use it and highlight some interesting real world use cases.  And there is a fully interactive Apple iBook edition of the pattern matching whitepaper coming soon! Stay tuned for more information! 

Technorati Tags: , , , ,

Tuesday, 25 June 2013

Southwestern Energy uses Oracle to deliver big data management

Another big data win for Oracle! Southwestern Energy chose Oracle for their big data management project to help them optimize reservoir characterization, drilling, and production operations by delivering an engineered systems platform to capture massive amounts of data, and make better operational, tactical, and strategic decisions. SWN is using Oracle's Engineered Systems Platform to capture massive amounts of data, and make better operational, tactical, and strategic decisions.

 

(If the video above is not visible or does not play then you can view the try following this link to the post on the Oracle Media Network: http://medianetwork.oracle.com/video/player/2431393994001)

For more videos from our big data customers explaining how they are using our solutions to drive their big data projects visit the Oracle Media Network by clicking here: http://medianetwork.oracle.com/search/results?q=big+data

If you want more information about big data in the oil and gas industry there is an excellent series of three presentations from OOW 2012 available on the OpenWorld site. The title of the session was "Big Data in the Oil and Gas Industry" and there were four presenters from three different companies:

  • Ali Al Ali - IT Division Manager, Adnoc-Dist
  • Tim Crowder - Principal Consultant, Chevron Ltd
  • Mohamad Akoum - Vice President (CIT), Abu Dhabi Company for Onshore Oil Operations
  • Awad Ahmed Ali El-Sidiq - Senior Database Administrator, ADNOC Distribution

Their presentation focused on the need to analyze massive amounts of data in real time in order to become more proactive. They looked at the need to reduce the lag time between data capture and data analysis, and between data analysis and the implementation of the resulting decisions. The key message from the session was that to become more proactive, oil and gas companies will need to eliminate the impact of disconnected and manual systems that prevent information from being used to act on proactive decisions. Companies will need to implement systems that enable them to capture real-time operational data, analyze the data to make “right-time” decisions, and execute these decisions seamlessly. 

The presentations from this session are here;

This session aligns perfectly with what we are now seeing in terms of how companies in this sector are implementing big data solutions. If you need more information about big data and how we can help you capture real-time operational data and analyze tate data to make “right-time” decision then please contact me via my blog.

Monday, 17 June 2013

Oracle Wins Big Data Company of the Year 2013

Great news from the Storage Awards 2013. Oracle won the "Big Data Company of the Year 2013" award. The event took place on 6 June 2013 at The Grand Connaught Rooms, Covent Garden. The aware has received by Yanhong Tong, Storage Marketing Manager at Oracle. 

DSC 5527

(picture from http://www.storage-awards.com/)

The 2013 Awards saw in excess of 11,500 readers of Storage Magazine vote for their favourite products, people and companies. Claiming the title of "Big Data Company" is great news for us because it tells us that our message of engineered systems to support big data along with big data tools to connect our Big Data Appliance to our Exadata data warehouse platform is the best solution to get customers up and running with big data and it allows them to maximise their analytical capabilities.

Well done Oracle Big Data team. For more information about Oracle's big data solution visit www.oracle.com/bigdata

Tuesday, 11 June 2013

My bike is monitored by Exadata…

I am a big fan of Garmin's sports gadgets and at the moment I have a Forerunner 210 on my mountain bike, a 310XT watch on my Trek 5200 and an Edge 705 loaded up with a map for my area of northern England on my Trek 5900SSL. Every morning I am either out on one of my road bikes if it is not raining or running with my 310XT watch if the weather is not so good. At the end of the week I connect both my Garmin devices to my computer and upload all the data I have collected to the Garmin Connect website so I can check on all the usual training statistics.  Turns out that I am actually using Exadata every single day! Garmin runs their Connect web application, along with its data warehouse and other operational systems, on Exadata. 

Here is a great video that really gets to the heart of why you want Exadata in your data centre and if you are Garmin user you will be pleased to know that every time you turn on your Garmin device you are actually using Exadata. 

(If the video above is not visible or does not play then you can view the try following this link to the post on YouTube: http://www.youtube.com/watch?v=jjB6mKYOr7M)

 Now, when I get up in the morning and press the power button on my Garmin device that startup process is going to be just a little bit more special. Great to know that Garmin and Oracle have got me covered, logging everything heart beat, calorie, pedal revolution, wattage and mile that I do each and every day. If you have never seen Garmin Connect then visit the website where you can take a guided tour of the features and check out all the latest Connect-enabled devices: http://connect.garmin.com/

Monday, 10 June 2013

Searching for insight and foresight...

We have just released a new video to support the recent report we sponsored by Economist Intelligence Unit research program. The video is an interview with Brian Eoff who is a senior data scientist at Bitly, New York. The new video is called "In Search of Insight and Foresight: Getting More Out of Big Data".   

Brian provides an interesting insight into the whole process of how he and his team explore the huge reservoir of data their company collects. The vast majority of questions that his team start out with seem to result in solutions and features that end up in the product although there are some cases where preconceived ideas have also been proven to be incorrect. 

One of the most important points raised by Brian is the much talked about disconnect between IT and the business teams which Bitly seems to have resolved by having an intermediary person who works with both sides. Admittedly this is nothing new but it does raise an important point which is that big data will not work in isolation so you have got to engage both sides (IT and the business units) if a big data project is going to be successful. If you want to understand some of the challenges of developing a big data project and some of the steps you can take to ensure it is successful within your company then this video provides some great points. 


If the video above does not work for you you can access it directly on the Oracle.com website by going here: http://www.oracle.com/us/corporate/features/economist-intelligence-report/index.html.

If you missed the report by the Economist Intelligence Unit that is mentioned throughout this interview you can download it from here:  http://www.oracle.com/us/corporate/features/economist-intelligence-report/index.html

Wednesday, 5 June 2013

Faster Data Movement from Hadoop to Oracle Database

The OTN team has just posted a great video interview with Oracle ACE Director Gwen Shapira on YouTube. Gwen talks about integrating Hadoop with Oracle Database using Oracle's Loader for Hadoop and she gives her views on when to use Hadoop vs. when to use Oracle Database.

If the embedded video above does not work for any reason then follow this link to the YouTube site: http://www.youtube.com/watch?v=tBpHAyTH6MU.

One of the interesting points that Gwen makes during the interview is the superior performance (5X faster) offered by our Hadoop loader compared to the other open source data loading tools. The Oracle Loader for Hadoop is part Oracle's Big Data Connectors suite of software and this suite of tools supports a wide range of Oracle targets. These connectors are designed and optimized to integrate Apache Hadoop with specific Oracle products including Oracle Database, Oracle Endeca Information Discovery and Oracle Data Integrator. The suite of connectors includes the following features/products:

FeatureBenefit
Oracle Loader for Hadoop

Uses MapReduce processing to format and load data efficiently into Oracle Database for analysis.

Oracle SQL Connector for HDFS

Enables Oracle Database to access data seamlessly from Hadoop Distributed File System allowing for SQL processing.

Oracle R Connector for Hadoop

Gives R users high performance native access to Hadoop Distributed File System (HDFS) and MapReduce programming framework.

Oracle Data Integrator Application Adapter for Hadoop

Simplifies data integration with Hadoop through easy to use interface

 

There is more information about the benefits of using the Loader for Hadoop to load data into your Oracle database in the recently published whitepaper High Performance Connectors for Load and Access of Data from Hadoop to Oracle Database.  For technical information about our Big Data Connectors, including links to download the software, visit the home page on OTN which is here: http://www.oracle.com/technetwork/bdc/big-data-connectors/overview/index.html. For more general information about big data goto www.oracle.com/bigdata