stream.php -Suchmaschine braucht hilfe

    Diese Seite verwendet Cookies. Durch die Nutzung unserer Seite erklären Sie sich damit einverstanden, dass wir Cookies setzen. Weitere Informationen

    • stream.php -Suchmaschine braucht hilfe

      Hallo Leute,

      ich brauche hilfe ,hier sieht ihr den Code einer Datei von meiner Suchmaschine. Sie sucht mit curL die Streams und trägt sie in der Datenbank ein. Und ich wollte gerne statt 2 seiten mehre seiten eintragen.

      Und onlinestreams und onlinemoviez wollte ich ganz weg haben.

      Folgendes Seiten wollte ich eintragen:





      Quellcode

      1. <?
      3. //var_dump(stream::lookup("rambo"));
      5. /*
      6. var_dump(stream::onlinestreams_org("rambo"));
      7. $url="http://www.onlinestreams.org/ajax/suche.php";
      8. $post="suche="."rambo";
      9. $ref="http://www.onlinestreams.org/index.php";
      10. $osorg=stream::curl_post($url,$post,$ref);
      11. preg_match_all("#<tr><td>(.*?)</td></tr></table>#ims", $osorg, $div);
      12. echo $div[1][0]." ".$div[1][1];
      13. die();
      14. */
      16. class stream{
      18. //searches db, if not found, scrapes and adds
      19. function lookup($title){
      20. $found=stream::search_title_db($title);
      21. if (count($found) == 0 || $found == "" ){
      22. //nothing in db, scrape it
      23. $found=stream::scrape_sites($title);
      24. if (count($found) == 0 || $found == "" ){return;}
      25. for ($i=0;$i<count($found);$i++){
      26. //watch out for other array structure as before!!!
      27. //$obj=$found[$i][2];
      28. $obj=str_replace('</center>','',$found[$i][2]);
      29. stream::add_db($found[$i][0],strtolower($found[$i][1]), $obj ,$found[$i][3]);
      30. }
      31. }
      32. return $found;
      34. }
      36. function scrape_sites($title){
      37. $erg1=stream::onlinemoviezzz_blogspot_com($title);
      38. $erg2=stream::onlinestreams_org($title);
      40. // FROM
      41. // array[0]=title, [1]=playerurl_object [2]=info
      42. // TO
      43. // url, title, obj, desc
      45. $j=0;
      46. for($i=0;$i<count($erg1);$i++){
      47. $comp[$j][0]=stream::extract_link($erg1[$i][1]);
      48. $comp[$j][1]=strtolower($erg1[$i][0]);
      49. $comp[$j][2]="<object>".str_replace('<param name="wmode" value="transparent"></param>','',$erg1[$i][1]);
      50. $comp[$j][3]=$erg1[$i][2];
      51. $j++;
      52. }
      53. for($i=0;$i<count($erg2);$i++){
      54. $comp[$j][0]=stream::extract_link($erg2[$i][1]);
      55. $comp[$j][1]=strtolower($erg2[$i][0]);
      56. $comp[$j][2]="<object>".str_replace('<param name="wmode" value="transparent"></param>','',$erg2[$i][1]);
      57. $comp[$j][3]=$erg2[$i][2];
      58. $j++;
      59. }
      60. //for additional scrapesites
      61. /*
      62. for($i=0;$i<count($erg3);$i++){
      63. $comp[$j][0]=stream::extract_link($erg3[$i][1]);
      64. $comp[$j][1]=strtolower($erg3[$i][0]);
      65. $comp[$j][2]="<object>".str_replace('<param name="wmode" value="transparent"></param>','',$erg3[$i][1]);
      66. $comp[$j][3]=$erg3[$i][2];
      67. $j++;
      68. }
      69. */
      70. return $comp;
      72. }
      75. //adds to db
      76. function add_db($url, $title, $object, $desc){
      77. if ($url == '' || $title == '' || $object == '' ) {return;}
      78. if (strlen($url) < 3 || strlen($title) < 3 || strlen($object) < 3 ) {return;}
      79. $link=mysql_connect('localhost', '', '') or die("Could not connect to MYSQL host");
      80. mysql_select_db('streamsearcher', $link) or die("Couldnt connect to databank.");
      81. $result = mysql_query("INSERT INTO data VALUES ('".$url."','".$title."','".$object."','".$desc."',1)");
      82. //if (!$result) { die("Couldnt send data to db: ". mysql_error());}
      83. }
      86. //searches db for title
      87. function search_title_db($title){
      88. $title=strip_tags($title);
      89. $title=trim(strtolower($title));
      90. $link=mysql_connect('localhost', '', '') or die("Could not connect to MYSQL host");
      91. mysql_select_db('streamsearcher', $link) or die("Couldnt connect to databank.");
      92. $result = mysql_query("SELECT DISTINCT * FROM data WHERE title LIKE '%".$title."%'");
      93. if (!$result) { die("Couldnt send data to db: ". mysql_error());}
      94. while ($i=mysql_fetch_array($result)){
      95. $out[]=$i;
      96. $i++;
      97. }
      98. return $out;
      99. }
      101. function titleonly_db(){
      102. $link=mysql_connect('localhost', '', '') or die("Could not connect to MYSQL host");
      103. mysql_select_db('streamsearcher', $link) or die("Couldnt connect to databank.");
      104. $result = mysql_query("SELECT DISTINCT title FROM data");
      105. if (!$result) { die("Couldnt send data to db: ". mysql_error());}
      106. while ($i=mysql_fetch_array($result)){
      107. $out[]=$i;
      108. $i++;
      109. }
      110. return $out;
      111. }
      115. //returns array[0]=title, [1]=playerurl_object [2]=info
      116. function onlinestreams_org($search){
      117. $url="http://www.onlinestreams.org/ajax/suche.php";
      118. $post="suche=".$search;
      119. $ref="http://www.onlinestreams.org/index.php";
      120. $osorg=stream::curl_post($url,$post,$ref);
      122. preg_match_all("#playMovie\('filme',(.*?)\)#ims", $osorg,$match);
      124. $url="http://www.onlinestreams.org/ajax/movie_play.php";
      125. $ref="http://www.onlinestreams.org/index.php";
      127. preg_match_all("#<tr><td>(.*?)</td></tr></table>#ims", $osorg, $titles);
      129. $found=array();
      130. $ret=array();
      131. for ($i=0;$i<count($match[1]);$i++){
      132. $found[]=substr($match[1][$i], strpos($match[1][$i], ",") );
      133. $post="id=".$found[$i];
      134. $ergsite=stream::curl_post($url,$post,$ref);
      137. preg_match_all("#<div (.*?)left: -40px\">#ims", $ergsite, $div);
      140. for ($j=0;$j<count($div[1]);$j++){
      141. $tdiv=$div[1][$j];
      142. $tdiv= substr( $tdiv, strpos($tdiv, "<param name="));
      143. $endp=strpos($tdiv, "</object>")+9;
      144. $purl=substr($tdiv, 0, $endp);
      145. $info=strip_tags(substr($tdiv, $endp));
      146. if (substr_count($info, "Megavideo") > 0) {
      147. $info=substr($info,236);
      148. }
      149. if (substr_count($info, "Veoh") > 0) {
      150. $info=substr($info,213);
      151. }
      153. $kat=array("Alle gemischt","Abenteuer","Action","Action-Komödie",
      154. "Anime","Drama","Fantasy","Horror","Kinderfilme","Kino","Kinofilme",
      155. "Komödie","Konzerte","Kriegsfilme","Krimikomödie","Melodram","Romantik",
      156. "Satire","Science Fiction","Special","Sport","Thriller","TVRiP's","XXX","Zeichentrick",
      157. "kategorie:","Kategorie:");
      159. for($p=0;$p<count($kat);$p++){
      160. $info=str_replace($kat[$p],"",$info);
      161. }
      163. //echo $purl."<br>\n".$info."<br><br>\n\n";
      165. $ret[$i][0]=$titles[1][$i];
      166. $ret[$i][1]=$purl;
      167. $ret[$i][2]=$info;
      168. }
      169. }
      171. return $ret;
      172. }
      176. //returns array[0]=title, [1]=playerurl_object [2]=info
      177. function onlinemoviezzz_blogspot_com($search){
      178. $url="http://onlinemoviezzz.blogspot.com/search?q=".$search;
      179. $ref="http://onlinemoviezzz.blogspot.com";
      180. $omz=stream::curl_get($url,$ref);
      182. $ret=array();
      184. preg_match_all("#Permanent Link to (.*?)'#ims", $omz,$titles);
      187. preg_match_all("#<embed(.*?)/>#ims", $omz,$match);
      188. $m1count=count($match[1]);
      189. for ($i=0;$i<$m1count;$i++){
      190. $m1="<embed".$match[1][$i];
      191. $endp=strrpos($m1,">");
      192. $m1=substr($m1,0,$endp+1);
      193. $thistitle= $titles[1][$i];
      194. $thistitle=str_replace("(","",$thistitle);
      195. $thistitle=str_replace(")","",$thistitle);
      196. $thistitle=str_replace("Veoh","",$thistitle);
      197. $thistitle=str_replace("(veoh)","",$thistitle);
      198. $thistitle=str_replace("Megavideo","",$thistitle);
      199. $thistitle=str_replace("megavideo","",$thistitle);
      200. $ret[$i][0]=$thistitle;
      201. $ret[$i][1]=$m1;
      202. }
      204. preg_match_all("#<div style=\"text-align: center; (.*?)</div>#ims", $omz,$match2);
      205. $m2count=count($match2[1]);
      207. if ($m2count != $m1count)
      208. {
      209. preg_match_all("#Beschreibung:(.*?)</div>#ims", $omz,$match2);
      210. $m2count=count($match2[1]);
      211. }
      213. for ($i=0;$i<$m2count;$i++){
      214. $m2=$match2[1][$i];
      215. $m2=preg_replace('#<center>(.*?)<\/center>#ims','', $m2);
      216. $m2=str_replace("color: rgb(0, 0, 0);\">","",$m2);
      217. $m2=strip_tags($m2);
      218. $ret[$i][2]=trim($m2);
      219. //echo $m2;
      220. }
      222. return $ret;
      223. }
      225. function extract_link($obj){
      226. preg_match_all("#src=\"(.*?)\" #ims", $obj, $erg);
      227. return $erg[1][0];
      228. }
      231. function curl_get($url,$ref="")
      232. {
      233. $ch = curl_init();
      235. curl_setopt($ch, CURLOPT_URL, $url);
      236. curl_setopt($ch, CURLOPT_REFERER, $ref);
      237. curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14" );
      238. curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
      239. curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
      243. $output = curl_exec($ch);
      244. curl_close($ch);
      246. return $output;
      247. }
      249. function curl_post($url, $params, $ref="") {
      250. $handle = fopen( "cookiejar.txt", "w+" );
      252. $ch = curl_init();
      254. curl_setopt($ch, CURLOPT_URL, $url);
      255. curl_setopt($ch, CURLOPT_REFERER, $ref);
      256. curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14" );
      257. curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
      258. curl_setopt($ch, CURLOPT_POST, true);
      259. curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookiejar.txt');
      260. curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookiejar.txt');
      261. curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
      262. curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
      266. $output = curl_exec($ch);
      268. curl_close($ch);
      269. @fclose($handle);
      270. return $output;
      271. }
      275. }
      277. ?>
      Alles anzeigen

    • wo ist denn konkret das Problem? Jede Seite hat ja ihr eigenes Regelwerk um die Items aus dem Quelltext zu extrahieren.
      Das musst du für deine neuen Seiten anlegen.

      Generell würde ich das ganze Klassenkonzept überdenken. Dein Konstrukt ist nicht erweiterbar.
      Am besten du definierst ein Interface mit den Standardmethoden eines Crawlers und implementierst das dann.

      Quellcode

      1. $this->crawler = array();
      3. function addCrawler(Crawler $c) {
      4. $this->crawler[] = $c;
      5. }
      7. function scrape_sites() {
      8. $return = array();
      9. forach($this->crawler as $c) {
      10. $return[] = $c->crawl();
      11. }
      12. return $return;
      13. }
      Alles anzeigen


      Desweiteren solltest du die Methoden nicht statisch nutzen.