关于SAE(Sina App Engine)对snoopy的支持问题

SAE支持fsockopen,那么snoopy现在也支持了。主要的测试项目如下:

1.利用snoopy获取指定的url页面

测试的链接地址:http://44.lazypeople.sinaapp.com/fetch.php测试的php代码为:

<?php
// start test
echo "i am alive!";
include("snoopy.class.php");
$snoopy = new Snoopy;
$url = "http://www.baidu.com";
$snoopy->fetch($url); //获取所有内容  
echo $snoopy->results; //显示结果
?>

主要的问题是编码问题,例如在抓取 手机新浪网就出现了乱码。

2.利用snoopy提交表单

测试的地址是:http://44.lazypeople.sinaapp.com/sendform.php (提交表单)http://44.lazypeople.sinaapp.com/form.php (接收表单),每次提交,接收端触发一次,利用KVDB存储的键值会自动加1。sendform.php代码:

<?php
// start test
echo "i am alive!";
include("snoopy.class.php");
$snoopy = new Snoopy;
$formvars["username"] = "saetest";  
$action = "http://44.lazypeople.sinaapp.com/form.php";//表单提交地址  
$snoopy->submit($action,$formvars);//$formvars为提交的数组
echo $snoopy->results; //获取表单提交后的 返回的结果  
?>

接收端代码

<?php
$kv = new SaeKV();
$ret = $kv->init();
$re = $kv->get('abcsss');
$re++;
$kv->set('abcsss',$re);
var_dump($re);  
?>

3.抓取页面的链接fetchlinks

<?php
include("snoopy.class.php");
$snoopy = new Snoopy;
$url = "http://weibo.com";
$snoopy->fetchlinks($url);
echo $snoopy->results;
?>

4.仅仅抓取文字链接fetchtext

<?php
include("snoopy.class.php");
$snoopy = new Snoopy;
$url = "http://www.baidu.com";
$snoopy->fetchtext($url);
$snoopy->results;
?>

5.利用snoopy的proxy功能

测试的地址是http://44.lazypeople.sinaapp.com/proxy.php 代理是百度搜素的,用的是50.18.187.62:8080,测试代码是:(包含了大部分的函数使用)

<?php
include("snoopy.class.php");
$snoopy = new Snoopy;
$snoopy->proxy_host = "50.18.187.62";
$snoopy->proxy_port = "8080";
$snoopy->agent = "(compatible; MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98)";
$snoopy->referer = "http://www.microsnot.com/";
$snoopy->cookies["SessionID"] = '238472834723489l';
$snoopy->cookies["favoriteColor"] = "RED";
$snoopy->rawheaders["Pragma"] = "no-cache";
$snoopy->maxredirs = 2;
$snoopy->offsiteok = false;
$snoopy->expandlinks = false;
$snoopy->user = "joe";
$snoopy->pass = "bloe";
if($snoopy->fetchtext("http://www.phpbuilder.com"))
{
while(list($key,$val) = each($snoopy->headers))
echo $key.": ".$val."<br>\n";
echo "<p>\n";
echo "<PRE>".htmlspecialchars($snoopy->results)."</PRE>\n";
} else {
   echo "error fetching document: ".$snoopy->error."\n";
}

6.snoopy类下载地址

点击下载

  1. Pingback: Sina App Engine(SAE)入门教程(8)- SaeFetchurl使用 | Footprints

  2. Pingback: Gregory Smith