Modify data from rss feed before inserting?

[dinki]

I've discovered that using news.google.com and an advance search I can get extremely good results for the type of news I'm trying to put on my site. The problem I am having is that the google rss feed has some extra information that I'd like to get rid of. Take a look at an example:

<item>

<title>Indianapolis Colts lead ESPY nominations - Times Picayune</title>
	
<link>http://news.google.com/news/url?sa=T&ct=us/2-0&fd=R&url=http://www.nola.com/newsflash/sports/index.ssf%3F/base/sports-13/1182776353191090.xml%26storylist%3D&cid=1117574525&ei=CRyARvatAYiy0AH_vtE0</link>

<guid isPermaLink="false">tag:news.google.com,2005:cluster=429cd57d</guid>

<pubDate>Mon, 25 Jun 2007 13:14:06 GMT</pubDate>

<description><br><table border=0 width= valign=top cellpadding=2 cellspacing=7><tr><td valign=top class=j><a href="http://news.google.com/news/url?sa=T&ct=us/2-0&fd=R&url=http://www.nola.com/newsflash/sports/index.ssf%3F/base/sports-13/1182776353191090.xml%26storylist%3D&cid=1117574525&ei=CRyARvatAYiy0AH_vtE0">Indianapolis Colts lead ESPY nominations</a><br><font size=-1><font color=#6f6f6f>Times Picayune,&nbsp;LA&nbsp;-</font> <nobr>6 hours ago</nobr></font><br><font size=-1>The Arthur Ashe Courage Award is presented to individuals whose contributions transcend <b>sports</b>. The new Jimmy V Award for Perseverance will be presented to <b>...</b></font><br></table>
</description>

</item>

I am using title=link_title, link=link_url, and summary=link_content . Unfortunately, each one of these is somewhat problematic.

For link_title, I would like to remove the '- Times Picayune' portion. Is there a way to get rid of this using php before it is submitted? The '-' is always there followed by the source.

For the link_url I'd really like to use the portion between 'url=' and '&cid'

Lastly, link_content I would like to keep what's between '<font size=-1>' and '</font>'

Is there any hope in hacking this together?

[ChuckRoast]

Is there any hope in hacking this together?

Pligg rss importer uses Magpie.

I would suggest researching Magpie forums. They might be able to help you..

[dinki]

Thanks for the tip. I've posted to their mailing list. I'll report my findings on this thread in case someone else is interested in doing something similar.

[P1mpPanther]

please do - i'd be interested in stripping a few things from my feeds as well.

[dinki]

I think I may have figured out some php that will reformat the data the way I want. Can someone tell me what file and perhaps what location in the file where I can grab the data and modify it before it is stored in the db?

[ChuckRoast]

I think I may have figured out some php that will reformat the data the way I want. Can someone tell me what file and perhaps what location in the file where I can grab the data and modify it before it is stored in the db?

I'm not 100% sure but I believe the file your looking for would be located in
domain.com/modules/rss_import/magpierss/rss_cache.inc

[dinki]

I was hoping to intercept and modify the data right before it is sent to the database rather than doing it on the frontend with magpie. I'm assuming that's what you're suggesting. If I can do it as late as possible then the data would be fully tested for duplicates/uniqueness through the rss importer.

I was hoping that I'd be able to find the spot where the data is inserted but alas I can't find it. My php skills are just good enough to shoehorn things into fitting. Hopefully it's just a matter of finding the right spot.

Thanks for the tips. Any additional help is really appreciated. I'd like to put a tip in the tip jar if I can get this thing figured out.

[dinki]

Many thanks go to Savant for pointing me in the right direction. Editing rss_import/templates/import_fields_center.tpl is all that needs to be done. I made a seperate function for 'fixing' the formatting of the title, link and content. Then I called the function for each variable after the variable was assigned. So far so good. Thanks again.

[dinki]

Can anyone tell me how to turn off caching? I can't get updates to happen any more often than every hour

[savant]

caching: pligg currently doesn't implement any caching with rss feeds.