PHP Classes

File: scrapper.php

Recommend this page to a friend!
  Classes of azizny   URL and Email Scrapper   scrapper.php   Download  
File: scrapper.php
Role: Example script
Content type: text/plain
Description: Scrapper Implementation
Class: URL and Email Scrapper
Scrape URL and e-mail addresses from Web pages
Author: By
Last change:
Date: 15 years ago
Size: 1,209 bytes
 

Contents

Class file image Download
<?php
/**************************************************/
/*
Released by AwesomePHP.com, under the GPL License, a
copy of it should be attached to the zip file, or
you can view it on http://AwesomePHP.com/gpl.txt
*/
/**************************************************/

/*
scrapper class implementation
*/

/* Include scrapper class */
include('scrapper.class.php');

/* Start a new scrapper object */
$do = new scraperStart;

/* Set maximum pages to scrape */
$do->setOptions(50);

/*
Set file locations and separators
setFile(emailFileLocation,urlFileLocation,separator)
*/
$do->setFile('emails.txt','urls.txt',"\n");

/*
Only do certain extensions
*/
$do->doOnly('htm');
$do->doOnly('html');
$do->doOnly('php');
$do->doOnly('asp');
$do->doOnly('jsp');

/*
Only do certain domains
*/
$do->onlyDomain('forums.digitalpoint.com');
$do->onlyDomain('google.com');

/*
Exclude the following domains
*/
$do->excludeDomain('yahoo.com');
$do->excludeDomain('ask.com');

/*
Start scrapping at this URL
*/
$do->startScrape('http://forums.digitalpoint.com/forumdisplay.php?f=24');

/*
Now store the begotten information into the files
*/
$do->storeList();

?>