<pedrocorreia.net ⁄>
 

<Scraping websites with Zend_Dom_Query ⁄ >




clicks: 3661 3661 2008-10-19 2008-10-19 goto programacao myNews programacao  Bookmark This Bookmark This


Today I stumbled upon an interesting and reportable scenario were I had to extract information of the weekly published Drum and Bass charts provided by BBC 1Xtra. As this information currently isn't available in any consumer friendly format like for example a RSS feed, I had to go that scraping route but didn't want to hustle with a regex approach. Since version 1.6.0 the Zend_Dom_Query component has been added to the framework mainly to support functional testing of MVC applications, but it also can be used for rolling custom website scrapers in a snap. Woot, perfect match!

The following code snippets are showing the Bbc_DnbCharts_Scraper class I came up with and an example of its usage. The class utilizes curl to read the website holding the desired data, which will be passed to Zend_Dom_Query to execute queries upon it. For querying the former loaded XHTML Document Object Model it's possible to either utilize XPath or CSS selectors. So I had to pick my poison, and decided to go with the CSS selectors as them were best suited for the document to query and will be more familiar to most jQuery or Prototype users. The query returns a result set of all matching DOMElements which are further unpuzzled via a private helper method returning just the desired charts data as shown in the closing listing. As you can see the implementation of the scraping can be done with a minimum of effort and these are exactly the moments I love the Zend Framework for.



este é só um excerto do artigo, para aceder ao artigo completo, clique no link em baixo:
this is just a small excerpt from the article, to access the full article please click in the link below:

http://raphaelstolt.blogspot.com/2008/10/scraping-websites-with-zenddo...




Subscribe News RSS  Subscribe News Updates by E-mail





myNews <myNews show="rand" cat="programacao" ⁄>

RouterJs: easy routing for your ajax Web applications new ...

RouterJs is a simple router for your ajax web apps. It's build upon History.js which means that Rout (...)

clicks: 17315 17315 2012-05-14 2012-05-14 goto url (new window) haithembelhaj.g... goto myNews programacao


Backbone computed properties new ...

This gist shows one way to implement read- and write-enabled computed properties on a Backbone Model (...)

clicks: 17101 17101 2012-05-13 2012-05-13 goto url (new window) https://gist.gi... goto myNews programacao


Android Query new ...

Android-Query (AQuery) is a light-weight library for doing asynchronous tasks and manipulating UI el (...)

clicks: 17219 17219 2012-05-12 2012-05-12 goto url (new window) code.google.com... goto myNews programacao


Create Instagram Filters With PHP new ...

In this tutorial, I'll demonstrate how to create vintage (just like Instagram does) photos with PHP (...)

clicks: 16983 16983 2012-05-12 2012-05-12 goto url (new window) net.tutsplus.co... goto myNews programacao


HTML5 jQuery Paint Plugin new ...

Websanova Paint is a HTML5 canvas based jQuery plugin. It allows you to free paint on a canvas area (...)

clicks: 28069 28069 2012-05-12 2012-05-12 goto url (new window) websanova.com/t... goto myNews programacao


Sass vs. LESS vs. Stylus: Preprocessor Shootout new ...

CSS3 preprocessors are languages written for the sole purpose of adding cool, inventive features to (...)

clicks: 16692 16692 2012-05-11 2012-05-11 goto url (new window) net.tutsplus.co... goto myNews programacao


Real-time Applications With Node.js and Socket.IO new ...

Hey everyone! Sorry about the long pause since the last blog post, life has been quite hectic for th (...)

clicks: 17353 17353 2012-05-11 2012-05-11 goto url (new window) codingcookies.c... goto myNews programacao


15 Handpicked jQuery Drop Down Menus Tutorials new ...

Here we are presenting another brilliant collection of 15 jQuery navigation menu that you can downlo (...)

clicks: 16124 16124 2012-05-10 2012-05-10 goto url (new window) smashingapps.co... goto myNews programacao