View Issue Details

IDProjectCategoryView StatusLast Update
0005889mantisbtsqlpublic2005-07-23 02:14
Reporterpeyn Assigned Tothraxisp  
PrioritynormalSeverityfeatureReproducibilityN/A
Status closedResolutionfixed 
Product Version1.0.0a3 
Fixed in Version1.0.0rc1 
Summary0005889: What's with all these queries
Description

Hi
I was wonderring what is going on with all these queries - are they all necessary?
Just on the first page, login_page.php, there are:
44 total queries executed.
16 unique queries executed.

I haven't looked at code but are ale these queries really necessary? On other pages there is also a significant amount of queries executed what makes mantis 1.0.0a3 work much much more slower than the old 0.18

TagsNo tags attached.

Relationships

related to 0005927 closedthraxisp null checking failure in user_get_accessible_subprojects (user_api.php) 
related to 0005928 closedthraxisp Pre-cache the available projects speeds up every page significantly 
child of 0005460 closedvboctor Critical Issues to Fix for Mantis 1.0.0 Release 

Activities

thraxisp

thraxisp

2005-07-07 12:27

reporter   ~0010687

Most of the new queries are related to the move of configuration to the database (0004416). There are also a number of new features (e.g., subprojects, name sorting), that may make it appear slower. The config change has a very small impact (<5%) in the benchmarking I have done.

MattB

MattB

2005-07-12 10:43

reporter   ~0010737

We're running a single Mantis MySQl DB (in UK) with two Mantis frontends (in UK and US). We do this because we have two domain controllers, one in US and one in UK and each local version of Mantis authenticates with the domain controllers but uses a single database of issues. The offices are connected via a VPN link running at about 2Mbits/sec both up and down. Yep it's complicated but works very well, apart from the speed.

The US site runs really slowly because of the queries - for example the new account page uses 144 queries and all it does is show the menu and a form to enter some details. I had a brief look into why all the queries are being run so often and it looks like the configuration system is pretty heavy on SQL use. When I get chance I'm going to look into it further to see if there's anyway it can be speeded up.

peyn

peyn

2005-07-12 14:36

reporter   ~0010739

Hi

As of what I've seen this configuration reading from DB is a nightmare. (You can turn on option to show all queries that are sent to DB - in config.inc). I did so and I don't know why but for the login page there are many queries like (example):

SELECT type, value, access_reqd FROM mantis_config_table WHERE config_id = 'top_include_page' AND project_id=0 AND user_id=0 ORDER BY user_id DESC, project_id DESC

SELECT type, value, access_reqd FROM mantis_config_table WHERE config_id = 'allow_signup' AND project_id=0 AND user_id=0 ORDER BY user_id DESC, project_id DESC

SELECT type, value, access_reqd FROM mantis_config_table WHERE config_id = 'lost_password_feature' AND project_id=0 AND user_id=0 ORDER BY user_id DESC, project_id DESC

And what's even worst these setting are read on every page.

Why isn't this done by one query

SELECT type, value, access_reqd, config_id FROM mantis_config_table WHERE project_id=0 AND user_id=0 ORDER BY user_id DESC, project_id DESC

and why isn't this done once (stored in some kind of $_SESSION['config'] var) - I think this wouldn't put that much of a pressure on DB.

thraxisp

thraxisp

2005-07-12 16:12

reporter   ~0010740

I have been testing a fetch algorithm similar to what you suggest. It is marginally faster by fetching all of the config variables in a single query. The drawback is that the memory caching algorithm gets slower because the selection of the correct value must now be done in php code (rather than the compiled SQL query). There are 3-5 thousand calls to "config_get" per page because of all of the flexibility we have built in. Currently only the first one results in a database hit.

peyn

peyn

2005-07-13 02:26

reporter   ~0010747

Last edited: 2005-07-13 02:27

Hi

I looked into the config table (there is no record? y?) and don't know what are access_reqd, type fields. I also don't know how did you tested. But ;-) just an idea (maybe it's obvious and you've tested it, maybe not so I'm gonna posted)

$q = "SELECT value, config_id FROM mantis_config_table WHERE project_id=0 AND user_id=0 ORDER BY user_id DESC, project_id DESC";
$result = mysql_query( $q );
while( $row = mysql_fetch_array( $result ))
{
$_SESSION[ 'config' ][ $row[ 'config_id' ]] = $row[ 'value' ];
}

And then in application:

if( $_SESSION[ 'config' ][ 'some_setting' ] )
...

I'm aware its not like I've invented a wheel or something but I had to posted just to have clean concious :D

best regards

MattB

MattB

2005-07-13 09:07

reporter   ~0010753

thraxisxp, if the DB server and web server are 4000 miles away over a 2MBit VPN link every tiny bit of optimisation makes a huge difference!

For example removing DB config checking completely cuts down page response time from 25 to 4 seconds. I started work on pulling the entire cnofig out in one go yesterday but didn't have chance to complete it - if you could do this and incorporate into the next release of Mantis I'd be extremely grateful!

0005927 and 0005928 also make huge speed improvements for our system.

thraxisp

thraxisp

2005-07-15 21:47

reporter   ~0010799

Fix submitted to CVS.

  • fetch all database entries into cache on the first call
  • process config heirarchy in php
  • cache results of config_eval for global variables
  • remove superfluous config_flush_cache calls

core/config_api.php -> 1.32
manage_config_workflow_page.php -> 1.13
manage_config_work_threshold_page.php -> 1.10
manage_config_email_page.php -> 1.8