View Issue Details

IDProjectCategoryView StatusLast Update
0027113mantisbtsqlpublic2021-03-07 18:28
Reporterfrankyb Assigned Todregad  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version2.20.0 
Target Version2.25.0Fixed in Version2.25.0 
Summary0027113: Error in bug_api.php when UPDATEing a bug
Description

There seems to be an error in the bug_aphi.php in function update(...)

The db_param() that is counted up while constructing the SQL statement is reset at parameter 18 so from there on it is counted a 1 again. This leads to the mantis-DB to whine about "integer is not a string".

This funktion is called in a plugin named "Source Integration" in the following way:

  • bug is retrieved from DB
  • bug is updated
  • bug is written to DB via bug_api.php::update()

Now even if I skip Step "bug is updated" and write back the bug that has bee read by bug_api.php via $t_bug = bug_get( $t_bug_id ); and write it back immediately via $t_bug->update(); the error is thrown.

Steps To Reproduce
  • Install MantisBT
Additional Information

Originally reported in https://github.com/mantisbt/mantisbt/pull/1686

TagsNo tags attached.

Activities

dregad

dregad

2020-07-29 13:00

developer   ~0064200

I tried my best to reproduce the problem, despite the lack of detailed instructions, but everything seems to work as expected.

  1. fresh install of mantis on PostgreSQL + Source Integration plugin configured for Github
  2. setup a test repository on GitHub, configure SI to resolve fixed issues
  3. Tried several ways to update an issue - from Mantis UI, by importing commits, and by pushing to the repo with GitHub webook setup.

This is the stack trace from the GitHub webhook test, e.g. http://example.com/mantis/plugin.php?page=Source/checkin&api_key=secret

bug_api.php:711, BugData->update()
Source.API.php:442, Source_Process_Changesets()
checkin.php:94, include()
plugin.php:74, {main}()

and the generated SQL in bug_api.php / BugData::update()

UPDATE {bug}
                    SET project_id=$1, reporter_id=$2,
                        handler_id=$3, duplicate_id=$4,
                        priority=$5, severity=$6,
                        reproducibility=$7, status=$8,
                        resolution=$9, projection=$10,
                        category_id=$11, eta=$12,
                        os=$13, os_build=$14,
                        platform=$15, version=$16,
                        build=$17, fixed_in_version=$18,
                        target_version=$19,
                        view_state=$20,
                        summary=$21,
                        sponsorship_total=$22,
                        sticky=$23,
                        due_date=$24
                    WHERE id=$25

As you can see, the query parameters are properly incremented, without reset at target_version as you're reporting.

So I'm wondering if there is maybe some specific setup on your side that I don't know about, or if I missed something.

frankyb

frankyb

2020-07-29 14:15

reporter   ~0064201

Actually, This report was intended to be finished the next days, and therefore I thought it was set to private �

So far this is good, and it worked for me too.

The point where it went wrong was when I set one project in mantis to a target version. On this point the db_param() was reset to 1

Actually I set up two identical instances on another server where I had root access and like you the error did not happen until I also introduced a target version (1 or 1.0.0, didn't matter)

frankyb

frankyb

2020-07-29 14:23

reporter   ~0064202

I mean, I scanned through some SQLs on Mantis that used db_parm() and this particular one that went amok on me was the only one that did a dB access in the middle of its building algorithm. That made me suspicious, put that acces in the start of the function and the SQL worked.

dregad

dregad

2020-07-30 03:41

developer   ~0064206

I thought it was set to private

Private means that issue is not visible to general public, but developers can see it.

I set one project in mantis to a target version.

Not sure what you mean by that, target version applies to individual issues, not projects... Can you clarify ?

put that acces in the start of the function and the SQL worked

Obviously that works around the problem, but I'd like to understand the root cause because I expect MantisBT core to push the current query param number onto a stack, and restore it after the middle query has been executed so numbering should continue where it left off (as it correctly does in my testing so far).

frankyb

frankyb

2020-07-30 09:40

reporter   ~0064211

In "Manage Projects" I added a Version to a project, then the Update query went wrong on me

dregad

dregad

2020-07-31 06:22

developer   ~0064212

In "Manage Projects" I added a Version to a project, then the Update query went wrong on me

OK I did that too, but still everything works fine for me.
Did you customize any code or install any plugins other than Source integration ? Also, please specify which VCS plugin you're using, even though I don't expect this to make any difference.

I think I'll need step-by-step instructions to reproduce, starting from a fresh installation.

frankyb

frankyb

2020-08-09 17:43

reporter   ~0064251

Thank you for your patience. I will write up a step-by-step on the next days on my private infrastructure.

frankyb

frankyb

2020-08-20 10:05

reporter   ~0064282

Last edited: 2020-08-20 10:06

Hi guys! I invested the whole day to reproduce the bug and the final outcome is:

It only happens in postgresql

If you like you can have a look into "reproduce.txt" but it boils down to

  • Use postgresl
  • In mantisbt go to Manage => Manage Projects => MyProject
  • Versions => 1.0.0
  • Add Version
  • Create Bug report
  • change file and commit it with fixed bug 0000002: test
  • git push
    ===> the bug is still open

So it seems to be a bug in the way the parameters are handeled differently in postgresql as opposed to mysqli it was working and I was looking very stupid out of my clothes when I tried to reproduce that in mysql :D

reproduce.txt (4,618 bytes)   
My steps to reproduce on Debian 10:
enable php like here:
https://tecadmin.net/install-apache-php-fpm-debian-10/
Install like here (I did not use the firewall ufw, too much fuzz on a local, temporary desktop machine):
https://www.howtoforge.com/how-to-install-and-configure-mantis-bug-tracker-on-ubuntu/
- Used 2.24.2 instead, and as tar, not zip
- dont use admin but root account for initial db creation (shouldnt make a difference, though)

edit /etc/gitlab/gitlab.rb and change external URL to external_url 'http://192.168.1.13:10443'

- create a project in mantisbt (mine is "Test")

Install gitlab: https://about.gitlab.com/install/#debian
Use Community Edition of gitlab, and I had to work around my companies https tinkerer like that (easy but evil way. I dont know why curl doesnt honor the certificates I've installed in /etc/ssl/certs/ca-certificates.crt):
curl --insecure https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sed 's/curl -/curl --insecure -/g' | sudo bash
EXTERNAL_URL="http://192.168.1.13:10443" apt-get install gitlab-ce

Create a new project "test"
Click on your Profile icon in the upper right corner to create an API Token:
- Settings (User Settings) => Access Tokens 
- Name: gitlab
- Scopes: tick all checkboxes
- Copy the access token data (mine was SBFDuUQY9YXXXXXXXbLE)

Install Source Integration Plugin from https://github.com/mantisbt-plugins/source-integration (https://github.com/mantisbt-plugins/source-integration.git master branch for mantisbt 2.x)
cp -r Source /var/www/html/mantis/plugins/
cp -r SourceGitlab /var/www/html/mantis/plugins/
chown -R www-data:www-data /var/www/html/mantis/plugins/Source*

Install Source Integration Plugin and its Gitlab child
(Just my experience: a thing I stumbled accross was that Source Integration Plugin was not in the list, instead there was the "VCS Basisintegration 2.4.0" which, on my first attempts I totally misunderstood.)

Configure Source Plugin
- tick all checkboxes under Enabled Features
- change Isses references RegEx (Pass 2) to /(?:fixe?d?s?|resolved?s?)+\s*(?:bug?s?)*+\s*:?\s+(?:#(?:\d+)[,\.\s]*)+/i
- generate API Key with openssl rand -hex 12 and copy that into field "API Key" and write it down for later
- click Update

Configure Project Archive
- Name: test
- GitLab Root: http://192.168.1.13:10443
- Gitlab Repository Name root/test
- Gitlab API Key from Gitlab: copy your acces token from above step here (mine was SBFDuUQY9YXXXXXXXbLE)
- click Update button

Configure Webhook in gitlab:
- Go to Admin area => Settings => Network
- click on "Outbound requests"
- tick "Allow requests to the local network from web hooks and services"
- click "Save changes"
- Go into your repository
- there on Settings => Webhooks
- insert http://mantis.eu.local/plugin.php?page=Source/checkin&api_key=YOURAPIKEYTHATYOUWROTEDOWN
- insert that one into "Secret Token Field" again - just in case Source Integration Plugin will switch on getting the secret token from Headers (that are encoded in ssl, unlike the URL which blows out the "secret" token into the www ;-))
- Tick every checkbox even though that is nonsense
- Untick "Enable SSL verification"
- click "Add webhook"
- check that it works: scroll down and click in the webhook's "Test" button and select "Push event" which hopefully yields a 200

Create a user that represents your git user in mantisbt

Create a "patch" in gitlab to get a note in mantis to our created bug #1:
- checkout your project git clone http://192.168.1.13:10443/root/test.git
- Create a new file. Name it test.txt
- Type in some text
- git commit test.txt Commit message is something like bug #1: test
- git push
- check that mantisbt has a note added to the issue
- edit your file again, commit with "fixed bug #1: test"  git commit test.txt
- git push
====> bug fixed

- Go to Manage => Manage Projects => Test
- Versions => 1.0.0
- Add Version
- Create Bug report
- change file and commit it with fixed bug #2: test
- git push
====> bug fixed

Install  postgresql and php postgresql connector
- apt install postgresql-11 php7.3-pgsql
- install mantis to postgresql as above, just select postgresql

Do exactly as above. Result is bug fixed until you add a target version, the bug #2 is not getting fixed!

Add my patch => bug fixed.

So in summary: with postgresql there seems to be a problem with the db_param getting reset somewhere in the middle of the queries, so my patch is not the right one. Postgresql database acces in mantis shoudl be fixed instead.




reproduce.txt (4,618 bytes)   
dregad

dregad

2020-08-20 10:28

developer   ~0064283

Thanks for the testing and thorough explanation and steps to reproduce. I will look in depth later, but here's some preliminary remarks:

  • This seems to confirm that there is a problem with the source integration plugin (and, more specifically, with the GitLab plugin as I was not able to reproduce the problem with GitHub)

  • Source Integration Plugin vs Source Control Integration vs VCS Basisintegration

    Source Integration Plugin is basically the "bundle" of the "main" (Source) plugin (which is called Source Control Integration) together with the VCS-specific child plugins. Anyway, this is basically a translation issue - if you feel the German wording is confusing, feel free to submit a PR with a better one.

dregad

dregad

2021-01-17 09:05

developer   ~0065004

Last edited: 2021-01-17 09:12

I think I finally nailed this bug.

Despite what I thought initially, it turns out it is not caused by the Source Integration plugin at all, but by an oversight in the DbQuery class, which did not consider that it could be used in parallel with legacy DB API queries and so failed to save and restore the state of the parameters stack.

The access_has_project_level() check, executes several SQL queries, to populate the User Prefs and Projects caches, via user_pref_cache_row() and project_cache_row(). It is the former that introduced the regression, in MantisBT 2.20.0, commit MantisBT master 16a46b50 (part of PR https://github.com/mantisbt/mantisbt/pull/1433)

dregad

dregad

2021-01-17 09:14

developer   ~0065005

PR https://github.com/mantisbt/mantisbt/pull/1732

Related Changesets

MantisBT: master 16a46b50

2019-01-01 09:55

cproensa

Committer: dregad


Details Diff
Refactor user_pref_api db queries

Refactor db queries that still use hardcoded parameters.
Affected Issues
0025850, 0027113
mod - core/user_pref_api.php Diff File

MantisBT: master 066f4bc8

2021-01-17 03:51

dregad


Details Diff
DbQuery: push/pop param count before execution

This is for backwards compatibility with legacy code still relying on
DB API functions. Without this, parameter count gets reset after query
execution, which causes issues on PostgreSQL.

Fixes 0027113
Affected Issues
0027113
mod - core/classes/DbQuery.class.php Diff File