RSS Importer - Interesting Dilemma with banning/discard- and solution/mod project

Register an Account
Pligg Chat Room
Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 04-26-2009, 01:18 PM
New Pligger
Pligg Version: 1 RC4
 
Join Date: Apr 2009
Posts: 10
For:
Pligg 1 RC4
RSS Importer 0.9

I'm using the RSS import for youtube videos and it works great. I do a series of searches based on a specific keyword. However, sometimes not all of the videos are relevant, so I want to remove them from the site when they show up and tidy up my site by only leaving the relevant content. Problem is, when I discard the link, it comes back next time RSS importer runs. So, what would be the best solution to fix this so that the video doesn't appear but the RSS importer still knows not to import it - it's still somewhere in the searchable database against what the RSS importer is grabbing (i.e. it recognizes the url so it skips it)?

I've seen people talk on the forums of using google reader to approve the stories to be sent to the RSS importer before the importer sees them. I'd rather not do this as it seems too time consuming to even get my stories imported and published. I'd rather filter out unrelated articles after they're already published at my leisure.

The ban domain option is the best one, but I don't want to ban all of youtube, just the link.

Step 1:

In 2 places in linkadmin.php (in the admin directory), there's this code

Code:
$main_smarty->assign('banned_domain_url',get_base_url($link->link_url));
change both of them to this:

Code:
$main_smarty->assign('banned_domain_url',$link->link_url);
There's 2 more problems to take care of now that this has been achieved, which I'm still trying to figure out and maybe someone can help me.

Problem #1:

Submitting a URL manually still causes the site to look at the base url in the local-antispam.txt file, not the whole url. It looks like I need to re-write the libs/link.php code, but I'm not sure where to change it. Any ideas?

Code:
   function _scan_url()
   {
       $req = $this->_url;

       $pos = strpos($req, '://');
       $this->_protocol = strtolower(substr($req, 0, $pos));

       $req = substr($req, $pos+3);
       $pos = strpos($req, '/');
       if($pos === false)
           $pos = strlen($req);
       $host = substr($req, 0, $pos);

       if(strpos($host, ':') !== false)
       {
           list($this->_host, $this->_port) = explode(':', $host);
       }
       else
       {
           $this->_host = $host;
           $this->_port = ($this->_protocol == 'https') ? 443 : 80;
       }

       $this->_uri = substr($req, $pos);
       if($this->_uri == '')
           $this->_uri = '/';
   }
Problem #2:

Then from there, I need to include a similar function into the RSS importer to scan local-antispam.txt for banned urls.

Problem #3:

After that, for an extra, I could add a button directly next to the link where the discard button is and make it so that this function of banning is easier than the 3 step process it currently is.

I'm gonna keep working on it and post my results. If anyone wants to chip in to help, that would be great too

Last edited by megaplatinum; 04-26-2009 at 01:26 PM.
Reply With Quote
  #2 (permalink)  
Old 04-26-2009, 02:53 PM
New Pligger
Pligg Version: 1 RC4
 
Join Date: Apr 2009
Posts: 10
UPDATE: This is not the best way to go about this! hehehe.

There's a basic bug in the submission in general - which I think affects the RSS importer. It only checks published news to search for duplicates. It does not check discarded news for duplicates. Any idea where this function is?

I figure, if I discarded a link, there's a reason I don't want it there, and hence it should not get republished, ever. That makes sense, doesn't it?

Where is this code? I'm going crazy trying to find it. Any ideas?
Reply With Quote
  #3 (permalink)  
Old 04-26-2009, 03:46 PM
New Pligger
Pligg Version: 1 RC4
 
Join Date: Apr 2009
Posts: 10
2 part solution:

I found this fix here:

http://forums.pligg.com/bug-report/1...discarded.html

To change it, we need to change these lines in libs/link.php

Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_url = '$link_url' AND link_status != 'discard'");
change to:

Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_url = '$link_url'");
and
and:
Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_title = '$link_title' AND link_status != 'discard'");
change to:

Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_title = '$link_title'");

part 2: when you click the "discard" button from the story itself, it actually deletes it from the database. We only want to discard it, so change this code in your template link_summary.tpl:

Code:
<a href="{$my_pligg_base}/delete.php?link_id={$link_id}">{#PLIGG_Visual_AdminPanel_Discard#}</a>
to:

Code:
<a href="{$my_pligg_base}/admin/linkadmin.php?id={$link_id}&action=dodiscard">{#PLIGG_Visual_AdminPanel_Discard#}</a>
Case closed. Problem solved! Works like a charm now!
Reply With Quote
  #4 (permalink)  
Old 04-26-2009, 09:12 PM
chuckroast's Avatar
Pligg Developer/Coder/Designer
Pligg Version: SVN
Pligg Template: Galleria
 
Join Date: Jun 2006
Location: PA
Posts: 3,828
I'm glad you got it figured out and decide to come back and share your results with the others. Interesting experiment and great results. Thanks again .


Get the full Pligg Module Pack today.

Anyone want the domain MyHubb.com ? Bid Now!




Reply With Quote
  #5 (permalink)  
Old 10-04-2009, 04:03 PM
kleinhev's Avatar
New Pligger
Pligg Version: 1.0.2
Pligg Template: wistie
 
Join Date: Jun 2008
Posts: 29
Quote:
Originally Posted by chuckroast View Post
I'm glad you got it figured out and decide to come back and share your results with the others. Interesting experiment and great results. Thanks again .
It is very basic stuff not to have duplicate URLs in your database. Since it is basic stuff, this should have been included in PLIGG since the very early days. How come it is still not incorporated in 1.0.0 and 1.0.1???
Reply With Quote
  #6 (permalink)  
Old 10-04-2009, 07:10 PM
Pligg Donor
Pligg Version: 1.0.1
Donation Level 1 
 
Join Date: Nov 2008
Posts: 111
Thanks for sharing! I had the same problem


Thanks again
Reply With Quote
  #7 (permalink)  
Old 10-05-2009, 03:09 AM
kleinhev's Avatar
New Pligger
Pligg Version: 1.0.2
Pligg Template: wistie
 
Join Date: Jun 2008
Posts: 29
Quote:
Originally Posted by megaplatinum View Post
2 part solution:

I found this fix here:

http://forums.pligg.com/bug-report/1...discarded.html

To change it, we need to change these lines in libs/link.php

Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_url = '$link_url' AND link_status != 'discard'");
change to:

Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_url = '$link_url'");
and
and:
Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_title = '$link_title' AND link_status != 'discard'");
change to:

Code:
$n = $db->get_var("SELECT count(*) FROM " . table_links . " WHERE link_title = '$link_title'");

part 2: when you click the "discard" button from the story itself, it actually deletes it from the database. We only want to discard it, so change this code in your template link_summary.tpl:

Code:
<a href="{$my_pligg_base}/delete.php?link_id={$link_id}">{#PLIGG_Visual_AdminPanel_Discard#}</a>
to:

Code:
<a href="{$my_pligg_base}/admin/linkadmin.php?id={$link_id}&action=dodiscard">{#PLIGG_Visual_AdminPanel_Discard#}</a>
Case closed. Problem solved! Works like a charm now!
Please re-publish this post since some parts of your post are not visible. I wonder why this substantial improvement has not been included in the official Pligg releases???

openSUSE 11.1 x86-64 | MySQL - 5.0.67 | PHP 5.2.11
Pligg 1.0.2 | (modified) wistie | Sciencia.org
Reply With Quote
  #8 (permalink)  
Old 10-05-2009, 10:14 AM
chuckroast's Avatar
Pligg Developer/Coder/Designer
Pligg Version: SVN
Pligg Template: Galleria
 
Join Date: Jun 2006
Location: PA
Posts: 3,828
Quote:
Originally Posted by kleinhev View Post
I wonder why this substantial improvement has not been included in the official Pligg releases???
You've mentioned this twice in the past 2 days. The RSS Importer is a Module and not part of the Pligg core. It's not the responsibility of the core developers to fix bugs in a module made by a 3rd party.


Get the full Pligg Module Pack today.

Anyone want the domain MyHubb.com ? Bid Now!




Reply With Quote
  #9 (permalink)  
Old 02-22-2010, 02:17 AM
New Pligger
Pligg Version: 1.0.3
 
Join Date: Feb 2010
Posts: 14
Just the solution i was looking for!

Thanks for your well documented work!
Reply With Quote
Reply

Thread Tools
Display Modes




Pligg Modules and Pligg Templates from Pligg Pro Find support on the Pligg CMS Forum - 24 hours a day! Make a donation to support Pligg CMS development