seems that caching and dbase calls are the impasse, sorry.

[daveyc]

so, i had a friend look over my pligg install and he knows quite a bit about sql (too much for his own good) - at first he was concerned about the number of queries and then realized that the template is not being cached (using yget on my site, running 9.1) and so all of these little hacks and tweaks (gzip et al) are in vain without real template caching...the entire thing is being redrawn each and every time a visitor moves around...(confirmed by ash: http://pligg.com/forum/showthread.php?p=22738)

in one scenario, the default pligg installation also leaves mysql persisten connection on (as 'true') - this means that one user could be sitting there for a few minutes using the 'capacity' of what 100+ users should be using...in turn, this puts every default install at risk of pissing off hosting providers if the sites actually get busy (most pliggs seem to have a couple of users, but a few will hit the thousands or more, eventually)...because most don't even understand how to edit db.php to set to 'false' it's a problem - but also setting to false makes the site even slower, redrawing templates AND running each query anew....but it should be downloaded with setting as 'false' so as to not push pligg users toward confrontations with hosting providers!!!

personally, if i knew enough about the underlying architecture, i'd be mad mad focused and only focused on caching templates IF you do not want to drop smarty templating...

...and the number of dbase calls is outrageous as wellof - for example, take a look at dragonflycms.com - a variation on nuke - it's extremely robust, and uses smarty (yes!) but only has 18 dbase calls from the home page, about 30ish for forums..versus 180ish for pligg showing a generic home page with like 10 stories...doesn't that trouble you a bit? i've done a few hacks via these forums to get those calls down to about 120, but that's still ridiculous, plus without caching there's no way to really effect performance and speed as a 'standard' (yes, some pliggs run super fast, particularly with awesome hosts, dedicated servers, etc)

has anybody attempted to hard code elements of layout to remove smarty and in turn enable caching, or cache with smarty, or reduce total queries or anything like that?

i'm only worried because i'm watching all of these people waste time modifying templates, removing little 4k images and waiting for the performance gains, or enabling gzip and waiting some more...without caching and reduced calls, the default will just always crawl for the majority of users.

please don't misunderstand me either - i love what pligg is doing, and i love the app (using it myself) - just wondering if it will ever be useful for a site that has any semblance of real traffic....

[kbeeveer46]

Have you tried the latest SVN? Ash has done a lot of work with speed and load issues. He was able to get the database calls down from 170+ to 40-60.

Here is a demo of the latest build: www.clubkb.com/pligg Maybe it's just me or the hosting but it's not slow at all.

[daveyc]

thanks for such a quick reply! i have not - running the 9.1 download though did visit the svn via another post from you and ash which took calls from (i think) 180ish to 120ish...what came after that? and how simple is the upgrade from 9.1 to the svn? could i simply not run an upgrade and overwrite a few specific files that he's worked on?

[daveyc]

btw, my own site is actually about as snappy as that kbclub site, so not ridiculously concerned (for now) - and i'm on mediatemple, so may adopt their new "mysql container" offering next month when it rolls out depending on pricing...and with that, i'll finally have access to a slow query optimizer and can then nail down which items are perhaps bogging things down (because it's certainly not all of them)

[Simon]

There were a number of changes in the latest svn as opposed to 9.1 - i've been following them myself and they have reduced query counts. I have also done a bit of hardcoding on my installation to reduce some of the functions (that contain queries) being called on the main pages.

I think Ash said he will be doing further optimisation shortly, so i'd hope to see the query count drop further.

Btw, maybe a nice debugging feature / plugin would be to display the number of queries per page (like Wordpress does for example)....oh wait there is one [ $db->num_queries ] - my front page is currently calling the database 41 times. It's not great, but even more established applications like wordpress can output more than that. If you add to that the "extras" like top stories and queued news (which I removed) then it all adds up.

[daveyc]

simon, thanks for that feedback - a couple of quick pligg newbie questions if you have a second: what did you remove (files/lines) to reduce the count so much? also, is there an easy way to track the query specific changes that are in the svn and implement them by simply editing those lines of code? i found two, but not sure where else to look...

[en3r0]

I know pligg is concerned about this as well, and keeps working at speed improvements regularly. It only gets better with each release though. Thanks for bringing this up, I had no idea so many connections were used.

[dollars5]

daveyc, I guess you are on the Grid server of MT. I know of a site that ran on similar settings with 9.1 and MT was able to handle it decently for a decent amount of users also it used <300 GPU for the month. I bet the new SVN code is lot better than the 9.1 and the work is in progress, this should use even less GPU for you. The mysql-container was still in beta and did MT open it up.

But as a personal suggestion, if you choose to go main-stream - I would suggest to use a dv server (as low as $50) of MT which is lot better than Grids.

[daveyc]

so mt is launching mysql containers first week of april - like the dv but without all of the assorted reseller features and panel tools (hence cheaper with four levels of service)...this should improve things quite a bit specific to dbase calls...

as for the svn, i'm still trying to find out what major changes were made and if i should over write some files OR just wait for the next release...since hte site is quiet right now, seems like waiting could be easier, unless the code changes are simple...

[Simon]

On the SVN changes, all I can suggest is looking for entries with "cache" in them. But no guarentee that will get all of them (there were a number of them as i recall, mostly made by Ashdigg).

On a related note, I was talking with savant yesterday, and he was just testing a datastore with pligg. Using a datastore should massively reduce query numbers for actions such as getting story data for the main page, as it will use 1 call to get all the data, rather than 1 + n stories calls. The downside is the 1 query will be a lot slower than the 9 quick calls (the saving on the front page with 8 stories was only 56ms) - however when the database gets busy, only having to call the database once will obviously be a big performance boost. I don't think this change will appear particularly soon, but it will be a good one!

One area we noticed a lot of queries were being generated on were comments. For example take a look at these 2 links:

http://news.oioplus.com/marketing/oi...ls-up-running/ - O comments, 14 queries
http://news.oioplus.com/politics/oio...ing-pligg-cms/ - 3 comments, 46 queries

Just wanted to give a heads up on that one, as with a lot of comments will rack up the queries very quickly. Looks like that will need optimisation, or caching, or both.