This is something that is an absolute prerequisite for me to migrate to Mantis. There's simply no way I can do so without keeping our existing bug history.

I'm interested to see that this worked for 0.19.a2. I had to change it substantially to work with 0.19.0. Right now I'm playing with it to make it a little more flexible. I'll upload it when I'm done.

Ultimately I will want to add the capability to import old text as bugnotes as well.

Looks interesting, but it needs some work. It needs to adhere to coding standards and the logic needs to be moved to a core file. Perhaps this feature also needs to be expanded a bit (preview?).

Anyone who wants to work on this?

P.S. Did the author give permission to include this?

I have almost completed a modified version of this program that will allow almost any field in an issue to be imported or defaulted or set to a constant value, and will submit it when I am done.

I've sent a note to the author of the original program asking for his permission to submit it.

This is the only issue in the database that I could find that mentions coding standards, so I have to ask: where are the standards documented? Or alternatively, I can submit what I've done and someone can add/change whatever is necessary for conformance.

Edit: completed, waiting for original author's permission.

Received a note from Cas Nuy (the original author):

"Good news, ofcourse you can. Looking forward in checking it out.

Kind regards,
Cas"

At this point, I can either attach the files I have or alter as required to comply with coding standards (which I would need access to). Which would the team prefer?

The coding standards are here:

http://www.mantisbt.org/guidelines.php

There is also a link to the more general PHP Standards.

This is great stuff gtomlin - keep it up! =)

Please look at the coding standards first. Sounds good though!

Note: the link to PHP coding standards on http://www.mantisbt.org/guidelines.php is obsolete. It points to http://utvikler.start.no/code/php_coding_standard.html, which in turn redirects you to http://alltasks.net/code/php_coding_standard.html. This last page says the domain alltasks.net is expired.

Updated version uploaded as mantis_import_issues.zip; I've tested this on 0.19.0 and 0.19.1. Anyone is welcome to use/misuse/abuse or modify it as they see fit. Have fun.

(thanks to Cas Nuy for the original idea!)

I moved it from 0.20.0 to 0.19.3, but it's too late for 0.19.2.

Great work on the patch. Had a question: Can this patch can handle importing into existing custom fields? I think that would be helpful if possible, especially when importing from another system.

No it can't as it's presently written. To do this in a generalized fashion would require interrogating the custom field definitions in the database and then constructing the import page on the fly based on what custom fields are defined, and I just didn't have time to do that. For my own needs, if I put more work into this it would be to import external text as bugnotes.

Thanks for the patch... However, I am having some difficulty getting it to work.
I get a successful import however, none of my data is properly imported... the summary and descriptions are coming up as empty/null when the importing code is calling the create_bug method... in the end, i basically get a lot of blank entries created for project_id = 0 in my bug table. Does any one have any ideas on what I am doing wrong ?

Thank you,
Ann

Try commenting the lines that delete the temporary import table, and use phpmyadmin (or equivalent) to have a look at the data in that table. This should help to narrow down the problem.

Thanks for the info... Actually, I figured it out. I had previously commented out the summary and description while I was debugging and for got to put them back in... my bad. It works great!

Thanks again,
Ann

I tested gtomlin's contribution (mantis_import_issues.zip) under 0.19.2 it works fine. I have two small issues:
1 - Fixed in Version field is imported in mantis_import table but not migrated to the system
2 - Build field is imported in mantis_import table but not migrated to the system

Results of investigation for item 2:
I identified one potential bug in import_issues.php file (I am not an expert) and needs to be validated:

Set up build

if($f_buildx == IMPORT_FROM_FILE) {

$t_build = $a_buildx[$i] ; (replace $a_buildx[$i] to $a_build[$i])

$t_build = $a_build[$i] ;

} else if ($f_buildx == IMPORT_USE_CONST_VALUE) {
$t_build = $f_const_build;
} else {
$t_build = '' ;
}

Sorry for the delay in getting back to this...bogged down with other work.

Scipion's suggested fix corrected the problem for the "build" field. and the "fixed in version" field problem has been fixed by using bug_set_field() after bug_create(), since bug_create() does not set fixed_in_version.

Before uploading the fixed files, I need some guidance from the team. In 1.0.0a(x), function print_manage_menu() in core/html_api.php tests authority for each menu item before printing a link to it. What authority do you think should be tested for importing issues?

Uploaded new version (mantis_import_issues_v2.zip). Tested on 0.19.0 and 1.0.0a1. Made an "executive decision" that import should require ADMINISTRATOR access.

This issue is marked as a child of 0.20.0, which is obviously never going to be a real release as we're at 1.0.0a[n] now. It's still in feedback status but it's not apparent what that feedback might be. What do we need to do to get this one back on track?

gtomlin, I didn't run the script yet, but I like the idea of a configurable importer.

• I wonder if this is a feature that is needed as part of the application vs. as part of the administration section (i.e. admin/ folder). Having it as part of the application is nice, but is it needed?

• You should be able to use user_get_id_by_name() from user_api.php rather than defining your own methods. Maybe this method wasn't implemented in 0.19.x. If that is the case, then you script should use the standard API if the function exists.

• When constructing the $t_column_list in import_issues.php consider creating an array of columns, then imploding the array into a string with the comma as the separator. This will simplify your code.

• Consider creating core/import_api.php and move as much functionality to the api.

• Use standard Mantis headers.

• Use function( x ) rather than function(x).

• Use $g_db_table_prefix and $g_db_table_suffix to name the table for the import.

• Given that you are processing the file sequentially, is it really of use to import the file into a table?

• Why do you read all issues in memory, then write all issues to the database. Why don't read one issue, then write it, and so on.

• You can add a menu option to the main menu using $g_main_menu_custom_options, we should probably implement a similar technique to add custom options to the manage menu. Although if your patch gets integrated into Mantis, then we won't need it.

• You can rename your custom_strings_inc.php to import_custom_strings_inc.php and require the users to include it from their custom_strings_inc.php or create a custom_strings_inc.php to include it from. Same for custom_constants_inc.php. This may simplify the updates for your patch, however, it is not needed if this becomes part of Mantis main distribution.

More to come later. I will need to set it up and do some testing.

will like to confirm the data will be imported to the current project view?
thanks.

fredckp, issues will be imported into whatever project is the current project when you start the import. If you are not in the desired project, switch before starting the import.

Thanks for the feedback Victor...I'll incorporate the bulk of your suggestions. A few comments:

• Should it be part of the application? I'd say so, since (a) you need to set up users, projects etc. before it is even reasonable to try importing issues, and (b) it could become necessary to import issues from another source at a later date long after the admin directory has been removed.

• core/import_api.php - I'll look into it. It's probably not important now but could be later if we start importing other things like users and projects.

• As far as the points regarding use of a table and order of reading and writing are concerned, these are things that I didn't change from the original contribution that I picked up. As it turns out, though, it proved handy sometimes to have the imported data in a table to look at it with phpMyAdmin after an unsuccessful import.

There was a discussion about implementing this as an administrator task or as part of the application. I appreciate implementing as part of the application because I will need this feature frequently. We often get Excel Sheets from our customers; it would be nice to import these and this should be done by someone with manager rights.

Just my humble opinion on that feature.

There are a couple of issues with this functionality. First of all, it doesn't work on the Safari browser (MAC OSX 10.4). But that might be a Safari thing, it does work with Firefox.

The more serious, showstopper bug though: it turns out that for recent versions of MySQL the LOAD DATA LOCAL INFILE was removed because of an apparent "security issue". After hours of searching I finally found the way to re-enable LOAD DATA LOCAL INFILE again.

Edit the file core/adodb/drivers/adodb-mysql.inc.php.
In it, find its own class definition. Change $clientflags like so:
var $clientFlags = 128;

Now I was finally able to upload the file, but alas, MySQL did not find my uploadfile. So now I'm stuck.

Error message: File 'bugs.txt' not found

Hi joostbaaij, Were you able to find a solution to the file not being found?
I was getting error: "The used command is not allowed with this MySQL version" (using MySQL 4.0.25-standard / PHP 4.3.11)
I then changed to $clientflags=128 as you suggested and then got the same error you now have: "File 'C:/Import_Columns_2.csv' not found (Errcode: 2)". If you have a solution please let me know. I'm using Mantis 1.0.0rc1 and import_issues_v2. Thanks!

The only way it is going to work, is upload the file to the server (via ftp/scp etc). It doesn't matter if you choose client/server the importer always looks on the server. sadly enough.

joostbaaij, where exactly the file needs to be uploaded on the server? Directory?

I have put the file on the server, in the same directory as Mantis, but can't get it to recognize the file. I click 'Start Import' and nothing happens, the page doesn't reload or anything. If I switch the path to a local drive 'C:/..' then it responds with the 'Can't find file error'. If you have an example syntax for a server path that you've found works it would help a lot.

My next step is to manually import the records into the Mantis_Import table the tool created (using PhpMyAdmin), then comment out the LOAD FILE part of the script, and retry running the tool from that point. Any other ideas?

There's not enough info in these recent posts to enable anyone to help.

What is the actual file location? How exactly are you typing it on the import issues page? What platform are you running? What MySQL version? What PHP version? What Mantis version?

For LOAD DATA LOCAL security considerations, see:
http://dev.mysql.com/doc/refman/5.0/en/load-data-local.html
The ability or inability to use LOAD DATA LOCAL is in the installation's hands.

Hi gtomlin, thank you for offering to help.

This is my setup:
OS: Windows XP
Browser: IE 6.0 (Tried FireFox 1.0.7 too)
MySQL: 4.0.25-standard
PHP: 4.3.11
Mantis: 1.0.0rc1
Import Tool: mantis_import_issues_v2.zip
File Location: Local PC drive
Actual File Path: 'C:\Import_Columns_2.csv'
Typing in form : 'C:/Import_Columns_2.csv'

Local Client Import Error Message:
Importing from: C:/Import_Columns_2.txt
Importing into project: Mantis Changes
Importing columns: (reporter,summary,description)
File 'C:Import_Columns_2.txt' not found (Errcode: 2)

I have never been able to import a file. Not from my local PC or from the server. When trying to load a file from the server the page stays as is after clicking 'start import'. It doesn't refresh or throw an error. Any help would be great, thank you!

I tested both client and server imports successfully and only after doing so I noticed that you are specifying 'C:/Import_Columns_2.csv' but the import is attempting to access 'C:Import_Columns_2.txt'. It looks to me like your system is doing something strange.

Thank you for taking the time to test it. I'll keep digging to see what the problem might be.

Updated version uploaded: mantis_import_issues_v3.zip. Addresses the following items in Victor's note 10322:

• use user_get_id_by_name() from user_api.php rather than defining a custom method

• simplified construction of column list.

• Use standard Mantis headers.

• Use function( x ) rather than function(x).

• Use $g_db_table_prefix and $g_db_table_suffix to name the table for the import.

• Process each issue as it is read from the temporary table rather than constructing arrays.

As I mentioned in my note 10359, it makes sense for the importer to be a part of the distribution, since it may be necessary to run it after the admin directory has been removed. Accordingly, I haven't done anything to the way the custom_constants_inc.php and constant_strings_inc.php files are updated.

BTW, is there a syntax to generate a link to a bugnote (analogous to # and an issue number to generate a link to an issue)?

gtomlin,

I've attached version 4 of the patch which includes the following:

1. Fixes for some notices that were generated by the code. Please turn notices ON to be able to see all errors in the code.

2. Used db* rather than mysql*.

3. Remove dieclean() since it was no longer used after using db*.

4. A couple of other minor issues (use file compare to make sure my fixes are correct).

I'd recommend the following:

1. Consider avoiding the use LOAD DATA INFILE since it seems to be disabled by default in MySQL. I think it will be much easier to use, setup, and more portable if you upload the text file, process it via PHP, delete the upload temporary file.

2. For the options of importing fields, it would be good to have an option for "default (if not exists)". This means that script should import the value in the text file, only if it is valid; otherwise it should use the default specified.

Note: Without changing MySQL settings, I could only get it to import a file from the server that is located in the database directory.

Error generated when importing a server based file on a MySQL 4.1.14-standard installed on Linux:

Database query failed. Error received from database was 0001045: Access denied for user 'username'@'localhost' (using password: YES) for the query: LOAD DATA INFILE "./testissues.txt" REPLACE INTO TABLE mantis_import_issues_table FIELDS TERMINATED BY ";" LINES TERMINATED BY "\r\n" (summary,description);

Importing the file from the browser machine worked fine. Is there a specific location on the server that the file should be placed?

The current situation is:

• On my Windows box, I only got it working using the server based file.
• On the Linux hosted environment, I only got it working using the user local file.

I just tested the mantis_import_issues_v4 on Mantis 1.0.0rc3 and the import worked without issue. As a non-programmer I have 2 queries that would really help if anyone had an answer.

1. Has anyone tried importing data including original bug id numbers? If you have managed this what code changes did you make?

2. Does anyone have an example of a custom field that they imported using this, what code changes did you need to make for this?

A couple of other comments:

• When importing a file from a windows client, the \ should not be replaced by /.

• Although the description field is not marked as mandatory, it is. I have changed the script to use the summary for description, if description is empty.

Victor, when you say that '\' should not be changed to '/' on a Windows client, what platform was MySQL running on? According to the MySQL 5.0 Reference Manual (see http://dev.mysql.com/doc/refman/5.0/en/load-data.html), "Note that Windows pathnames are specified using forward slashes rather than backslashes. If you do use backslashes, you must double them." My tests were done with client and server both on Windows boxes, and my results were:

• \ - file not found
• \ - file not found
• / - OK

I read the / \ comment and I thought I would share my experience.

I used mantis_import_issues_v4 on Mantis 1.0.0rc3 on a WinXP machine running MySQL 4.1.11. Importing a cvs file from c:\ and I found it necessary to change the \ to /.

If someone could advise me how to import custom fields using this I would really appreciate it.
Thanks AG

AG, this is just using bug_create(), which creates an issue without custom fields. To make this importer support custom fields would be a substantial project.

I've attached v5 which includes the following changes:

1. Uploads the file to the server temporary directory, then imports it from there.
2. Escapes the strings before using them in constructing SQL queries. This is to avoid SQL injection attacks.

This will probably break the server-side files. Do we really need to support such case?

I had a look at your v5 files, and the approach should be fine. The important thing is that the facility works for all users in all environments. The user just needs to know where the file needs to be located when he/she starts the process.

Isn't this something that it would be more sensible to use mantisconnect to do?
I'm thinking a 'php client which talks to the webservice' before anyone says anything about c#.

Importing Issues somewhat strikes me as something you'd do from a command line script, rather then a webpage - for example, I did start trying to use a c# client at work, to add issues to mantis based on the output of an xml file.... Personally, I'm thinking this sort of thing is something better done as a contributed script/addon, then as a 'core feature'

In terms of a critical review of issues in v5:

1. DROP TABLE IF EXISTS is mysql specific, and shouldn't be used.
2. CREATE TABLE uses VARCHAR(32) etc - this should use ADODB's type's instead.
3. Creating/dropping tables should be done as part of the installation, not as part of the script.
4. LOAD DATA INFILE / REPLACE INTO TABLE / FIELDS TERMINATED BY / LINES TERMINATED BY / LOAD DATA LOCAL INFILE is specific to mysql and shouldn't be used.
5. why is the table even needed in the first place? If someone uploads a comma seperated file - why not open it on the server, and do the parsing there, rather then importing it into a database and retrieving it again?
6. I think import_issues.php should call helper_begin_long_process();

While you were working on bug corrections, I developped the following features required at my work.

• Import custom fields
• Import "status" field
• Import "resolution" field
• Import "date submitted" field

Because some fields are set to default by bug_create, function bug_set_field is used to import extra fields (status, resolution, date submitted). A consequence is that the bug history is updated each time such a field is imported. Custom fields API, specially custom_field_set_value, is used to import custom fields values.

Unfortunatly, this is based on V3 version and all modifications till V5 have to be merged. Does anyone want to perform this merge? I will attach this version as Mantis_Import_Issue_v3.1.

Sorry, the archive V3.1 is just a copy of V3 :-(
Use V3.1_effective instead.

Why a reporter can't delete its own attached file in this page?

Thanks jblanchon for your contribution. I uploaded v6 which includes the following:

• I merged your changes (v3.1).
• Did some code cleanup.
• Some changes towards Mantis standards.
• Did some fixes relating to the importing of status, resolution, fixed in release.
• I didn't do any testing relating to custom fields.

I'm trying to import the sample "testissues.txt" file as-is and I get:
APPLICATION ERROR #11
A necessary field '' was empty. Please recheck your inputs.

Somebody (apparently) fixed the script so that you no longer have to modify the path on Windows systems (as i remember in older versions having to switch the slashes). Therefore, if you follow the now-misleading directions and painstakingly change all the backslashes, you will experience app error #11.

...So I adjusted: "custom_strings_inc.php" at line 18: "$s_import_instructions_2 = "2. Specify the input file (Windows users change all \ to / in file name) and its characteristics.";

... replacing this line 18 with:
"$s_import_instructions_2 = "2. Specify the input file and its characteristics (Some Windows users may need to change all '\' to '/' in pathname).";

I submit only the "custom_strings_inc.php" file since somebody should verify that the syntax above is not problematic on non-Windows systems (i'm not certain about the '\' and '/' part). Then it could be folded into the main distribution.

First, thanks for the great work!

I have applied a few changes to import_issues.php and hope that they are usefull. The reason was that I could not import data, receiving application error 11. Basically, in the section that creates the bug entries from the temporary table, columns are now referenced by key instead of by index number, and the loop has been changed to a for loop. The PHP Version used in my environment is 4.4.2, and Mantis is 0.19.3.

I don't know if I should supply a new version or if you, gtomlin, will incorporate my changes. Also, I'm not sure whether my changes will work in other versions of PHP etc. My changes are based on mantis_import_issues_v6.zip.

Please let me know how I can best submit my suggestions.

mgfeller, Victor and others have changed this script substantially since I last touched it (which was _v3). I don't presently have any time to work on this, and I wouldn't want everyone else waiting for me because the last note mentioned my user name!

Using version 6!

When I try to import a file I get this error message:
APPLICATION ERROR 11
A necessary field '' was empty. Please recheck your inputs.

Does it has to be txt file? I used a csv file.

I put my file on the server in the root of my Mantis directory and still I'm getting this error message.

What am I doing wrong??

MPA, try mantis_import_issues_v6.1.zip which incorporates my changes as well as custom_strings_inc.zip submitted by atomoid.

michael

I am using 6.1 with mantis 1.0.1. Not sure if that is supposed to be supported or not, but I get the:

APPLICATION ERROR 11
A necessary field '' was empty. Please recheck your inputs.

No matter what I try when using the import page. I tried importing only the 2 required fields, and setting all the rest to either default or expclitly setting, same error as above.

Is there a way to turn on some sort of logging besides "a required field was empty?" Maybe print out the name of the field that was empty? Or maybe that error message is just a red herring.

wade_albright, do you have a custom field that is required when submitting a new issue?

vboctor, no, not that i know of. i'm just using 1.0.1 out of the box.

Using Mantis version 1.0.1 with import v6.1 gives me the following error;

With the import file on the server in the Mantis root folder;
APPLICATION ERROR 0000401
Database query failed. Error received from database was 0001045: Access denied for user: 'parlitco@localhost' (Using password: YES) for the query: LOAD DATA INFILE "/tmp/phpNnxdiN" REPLACE INTO TABLE mantis_import_issues_table FIELDS TERMINATED BY ";" LINES TERMINATED BY "\r\n" (reporter,priority,severity,reproducibility,status,resolution,category,view_state,summary,description,
custom_field_2,custom_field_5,custom_field_9,custom_field_3,custom_field_4,custom_field_8);

It seams that I don't have autorisation for de DB. This seems not correct. I'm sure that I'm using the correct name and password. But what about localhost? MySQL is running on the server at my provider. Maybe this has got something to do with it?

And maybe someone can attach a demo import file which is working correctly, so I can check/test this out?

EDIT: dregad 2011-11-01 break long lines to avoid horizontal scrolling

I'm afraid I have tested mantis_import_issues_v6.1.zip only with Mantis 0.19.3. I hope to use Mantis 1.0.1. soon... and then maybe extend the error messages, if at all possible. The app err 11 usually meant (for me) that the parsing didn't work at all.

Are field delimiter and end of line marker ok? Does your data make it into the temporary table (result of LOAD DATA INFILE ...)?

Are field delimiter and end of line marker ok? Does your data make it into the temporary table (result of LOAD DATA INFILE ...)?

Yes, It seems that all other variables are ok. And no, the temporary table is not filled.

I'm getting very close!
I'm not sure if my import file is correct. Maybe someone can confirm this. In my import file i'm using the id's for the reporter, status, severity, view status, etc.
Like this; 10;20;10;30;10;etc.

Is it possible that someone can sent me a import example file, where every column is used, including which columns i have to set on 'import', 'default' or 'Set all to'?
I would appriciate that very much.

I tried the 6.1 version on a system where the mysql server runs on a windows machine, and the apache/php runs on a FreeBSD machine. I placed the import file on the windows machine, and checked the 'server based' checkbox.
This did not work. The gpc_get_file() function assumes that the file can be approached from the php/apache server, which was not the case. After changing file input to an ordinary text input, the script ran succesfully.

Unchecking the 'server based' check box gave a lot of problems with MySQL rights. I could not figure out how to handle this.

I'm trying to use the 6.1 version on a windows system with easyphp installed. Im having the following error message:

Error 401: database query failed.

I'm using a very simple file to test the import feature, i tried to manually launch the query "load data infile..." in the temporary table, it worked perfectly. Any idea where the problem comes from?

By the way i would need to do the import manually for some reasons, could anyone tell me how to build the queries i will need to do it?

I use the "issue import v6.1" with Mantis V1.0.3. On the mysql server the "local infile" option is enabled. Still I get an error:

Database query failed. Error received from database was #1148: The used command is not allowed with this MySQL version for the query: LOAD DATA LOCAL INFILE "/tmp/php20HwUP" REPLACE INTO TABLE mantis_import_issues_table FIELDS TERMINATED BY ";" LINES TERMINATED BY "\r\n" (reporter,summary,description);

How can I avoid this and import an issue?

Sorry I don't saw before but 0004220:0010919 solved the problem.

The custom field date type where imported as String.
I added few lines to convert the String date of the CSV to a real date type in the database:

August 2006 - mBernigaud : don't set empty date -->

if ( $t_custom_field_value != '') {

August 2006 - mBernigaud : Convert string date to its internal value if ( $t_def['type'] == CUSTOM_FIELD_TYPE_DATE ) {

$t_custom_field_value = strtotime( $t_custom_field_value ) ;
}
if( !custom_field_set_value( $t_id, $t_bug_id, $t_custom_field_value ) ) {
echo '

' . lang_get( 'import_custom_field_failure' ) . $t_def['name'] . ' :
' . $t_summary . '

I'm currently (9/12/2006), unable to download the files. They download to a 0k file. Not sure if this has to do with the recent domain change. Also, I'm hoping there are some instructions within the file about the installation and use. If not I plan to write some and will post them if desired by others.

carp3tshark ,

This probably ocurred when Mantis switched their nameserver or upgraded to 1.1a1
A similar thing happened to me as noted in this forum post: http://forums.mantisbugtracker.com/viewtopic.php?t=307
I already sent a note to Victor.

Hello

Where are attached files ?

Since this hasnt been fixed yet, i uploaded the most recent version i had,
which of course, isnt the most recent version.

I titled it "mantis_import_issues_v6.1--reuploaded.zip" since it wouldnt let me upload a duplicate name, anyway there it is...

in a pinch you can roll in the 6.1.1 fix noted in the post above: (0013188) mbernigaud

Hello:
I couldn't find this documented anywhere
What is the expected value for Status in an import file?
I've tried both the literal string
New
and the numeric constant
10
and both seem to get munged to zero (0) on import.

I found a problem in the 'mantis_import_issues_v6.1--reuploaded.zip'.
'View Status' is set to public even if I set 'Set all to' as private.

It was easy to fix.
The line 511 of import_issue.php

} else if ( $f_buildx == IMPORT_USE_CONST_VALUE ) {

should be

} else if ( $f_view_statex == IMPORT_USE_CONST_VALUE ) {

Does anyone have a 1.1.0a2 patch for this feature? Also, will it eventually be easy to use the same php extension to export issues to a file as well?

I've tried a multiplicity of different combinations trying to get the csv importer to work. It appears that it doesn't respect the order of the fields that I put in my csv file, based on what I can see from the error message. Here's my input file (containing 2 lines) which imports data for every single field in the 'import issues' page:

"reporter";"handler";"priority";"severity";"reproducibility";"status";"resolution";"category";"date_submitted";
"os";"os_version";"platform";"version";"fixed_in_version";"build";"view_status";"summary";"description";
"reproduce";"info";"custom_field_2";"custom_field_1";"custom_field_3"
"svermeul";"svermeul";low;"minor";"always";"closed";"open";"go";"1998-07-16";
"win";"xp";"win";"none";"tip";"0";"private";"Default numeric format seetting";"Description: test";
"how to reproduce";"more info";"date closed";"bug";"19838"

and here are the columns the importer says it's importing:
(reporter,handler,priority,severity,reproducibility,status,resolution,category,date_submitted,
os,os_build,platform,version,fixed_in_version,build,view_state,summary,description,
reproduce,info,custom_field_2,custom_field_3,custom_field_1)

but notice the error message:
error importing custom field PR_number : "Default numeric format seetting"

it's complaining about the last field in the input file, but the data that it says it received for that field is actually from a field in the middle of the line.

Would anyone have a sample record that actually works? Please post it here for everyone to see or drop me a line at stewart 'at' midwinter 'dot' ca.

thanks so much!

EDIT: dregad 2011-11-01 break long lines to avoid horizontal scrolling

BTW, I'm running Mantis 1.06 and mySQL 4.1.20.

midtoad, what happens if you remove double quotation marks for each fields?

removing the double quotations from all fields except those with commas in the text has no impact on the importer. It still appears to process the fields in the wrong order.

This is a blocker issue for me. I can't use Mantis if I'm unable to import my existing issues.

I have attached 'mantis_import_issues_v7.zip' which allows you to import 'target version' introduced in 1.1.0.
It also contains fixes reported in 0004220:0013188 and 0004220:0013839.

Notes:

• The 'target version' is displayed only in mantis 1.1.0a1 or later.
• The 'category' field is required in 1.1.0 (0007801).

Tested on:

• Mac OS X 10.4.9 / MySQL 5.0.27 (utf-8) / Mantis 1.0.6
• Mac OS X 10.4.9 / MySQL 5.0.27 (utf-8) / Mantis 1.1.0 (latest CVS)
• Debian 3.1 / MySQL 5.0.27 (utf-8) / Mantis 1.1.0 (latest CVS)

--
midtoad,
Sorry, I missed the point in my last comment.
You cannot specify the order of fields. See the usage section of the 'readme.txt'.

I use v6.1 import function and have fixed in it a bug relative to custom fields of type date : their values need to be converted to php internally format by
strtotime function.

See import_issue.php uploaded 2007-05-21 10:15 (sorry I forgot to suffix it
with v6.1)

Hi,

I have written a plugin to import/export data from/to Mantis. Import supports only XML format at this time but other formats will be supported soon.
http://deboutv.free.fr/mantis/plugin.php?plugin=importexport

I added a new importer inspired by the one posted by ave. This one allow to import CSV files, where columns are in any order you want. It also can update issues, instead of creating them if you have a bug id column. And finally, it does not use the MySQL load data functionality (which was not working in my firm), but only Mantis PHP functions, and therefore is compatible with any supported database.
See csv_import-1.0.tgz

Hope this can help.

There is an error in the script I uploaded in import_issues.php at line 117. This line should be:
$t_bug_exists = array_search( 'id', $f_columns ) !== false;

I just tested the Import but the import of custom fields dont work at all
I am using mantis 1.1.1.

All other data is imported correctly.
Any idea whats wrong?

I found the problem...
there is a little bug in the importissue.php
at line 246 the custom fiels are retrieved but without a custom
Thats why they cant be found later

For my needs I simply added a line

Look if this field is set

$t_def = custom_field_get_definition( $t_id );
$tdef['name']="custom".$t_def['name'];

We should consider re-developing this patch as a plugin for Mantis 1.2.x. I will look into it when I get a chance unless someone else beats me to it.

Is it possible to transform this tool that a note can be imported. Should have a note for this ID exist, a new note added.
Would it be possible to the text separator ',' or ';' return to the options presented?

Greeting
osterwaldi

I wanted to write something about the new file csv_import-1.0.1.zip,
but there occured an error in database_api.php as i tryed to send my text.

That is why the note I wrote was posted so often.
I'm sorry about this, i have deleted all previous notes of mine with same content.

uso

I have done some little work on the CSV Import plugin.
It is now able to work as plugin for Mantis version > 1.2.0.
Also I've implemented some features, like automatically adding handlers,
if they are listed in the CSV-File and do not exist in the database.

I made changes to CSV Import 1.0 only for my company,
intentionally it was not planned to release the changes.
But now I saw it helped me a lot, so I released it.

I hope it will help somebody, too. But do not treat me if it does not. ;-)

(I'm sorry to the developers, my coding style is not what it should be...)

uso

Dear Members,

I am so confused in choosing the right version of mantis.

I need import option in mantis 1.1.6, where the other plugins like myViewIssues,Hierarchy plugins are to be supported by this.....

Please do let me know, how to use and which one to use with mantis 1.1.6 to have the option to import in to CSV or XML format.

Is there anyway by means of customization, we could use the import/export plugin version 0.0.4 to use with mantis1.1.6

Regards,
Prabhu.

I'm also looking for an import excel feature and wondering if 1.1.6 has it.
I'd also like to suggest for consideration that an import feature keep this in mind.

1. Keep track of what was imported and check for duplication so we don't contaminate the core Mantis issues table with duplicate entries.

2. Take into consideration concepts like updating existing mantis issues if the imported data is an update to an existing issue based on some correlation field like say some unique ID from another system that Mantis is also tracking.

3. have an ability to determine what the default behavior is of certain fields such as status or category and whether or not an import where there is already an existin mantis issue on whether to update the field or let the existing field take precedence. One thing which can complicate matters is to possible code Mantis to determine if it's fields have the concept of Null, where the person entering in the issue did not know the value of the field versus it was a default value. Should an update update a null value? I think yes. Should an update update a defaulted value? It depends. If the defaulted value has a value like Unknown I think it should update that value because it is equivalent to Null, however, if a field has a value of Resolved for example, I may not want this third party import updating what I may have painstankenly figured out and changed manually.

4. This brings up the point of possibly having an intermediary view of potentially imported values where a test import could occur and a comparison view opens up showing a diff on the merge and allow a human to determine which field (either the existing mantis field or the imported field) should take precedence or the option of changing it all together.

Needed to import issues into 1.2 and adjusted this plugin to fit my needs.
Now no more manual action, also added the due-date column to be imported.
Version 1.1.0 at your service!

@cas

Cas will this CSV import work with mantis 1.2.0a3, please let me know. Already i have XML import contributed by you. Now please let me know how to use this CSV import with the XML import.

Regards,
Prabhu

The XML is not by me.
This one works with 1.2.0a3, no problems

Oh yes, Reminder is by you......

Dear All,

I have an issue with XML import; It is working fine in WINDOWS machine, but it throws some error in LINUX machine; Any one faced this issue.
I would like to know where the imported files are kept by mantis, while importing issues. Also Export feature is also not working in LINUX.

Or it would be a file permission in LINUX.

Error:
Error i faced is, The file could not be found at http://localhost/mantis/plugin.php=ImportExport/Import-action

Regards,
Prabhu.

Dear All,

I am facing this issue while trying to import issues using CSV Import:

Fatal error: Call to undefined function explode_enum_arr() in C:\Documents and Settings\prabhu_rangan\workspace\mylo\plugins\Csv_import\pages\import_issues.php on line 76

Please anyone let me know what is wrong with this.

Also which are the required fields to import CSV issues, to create new issues; like the attribute 'ID' should be specified '0'

Regards,
Prabhu

Add the following code @ the end of import_issues_inc.php (before the closing tag):

--------------------

# Given one num:value pair it will return both in an array
# num will be first (element 0) value second (element 1)
function explode_enum_arr( $p_enum_elem ) {
    return explode( ':', $p_enum_elem );
} 

Have updated my source also.

@cas

I am facing this isssue, do i have to add a function same as to 'explode_enum_arr()'

Fatal error: Call to undefined function explode_enum_string() in C:\Documents and Settings\prabhu_rangan\workspace\mylo\plugins\Csv_import\pages\import_issues.php on line 73

@cas

Sorry cas, its my mistake.

No issue

@cas

I have added a custom_field 'Deadline' and 'Est.' hours to my view issues and while importing issue to CSV, i need to import values to these two fields also, how shall i perform it.

In the '$g_all_fields' array i have mentioned my default columns for import, but after importing i could not see the values imported for these two fields.

Can you please share your ideas

Custom fields will show up in the list of available fiedls automatically.
Please do not change the code for this but ensure that the custom fields are connected to the project in which you are importinh.

it should bring up some error messages if type or content of columns do not match

in some cases like invalid values for category you do not get HTML at all

in that respect also two feature requests:

• create the entries in the column of the CSV as new possible values for that column

• create a column of the CSV as new custom column

apart from that: a real good tool; it makes mantis a lot more usefull than it already is, great work, thanks

BTW: does the CSV import has an own web page? maybe this plugin is enough of a project of its own so it should its own tracker

feature request:

match automagically columns with same name

advantage: you could prepare the import in the application that generates the CSV without matching each column manually during the import

even moving issues from one mantis system to another would be very easy that way

Although I started this tool in the past, GTOMLIN took over. I then converted the latest code into a plugin sinci I was excercising plugin development and used 1.2.
For me this tool is good enough since one would only use it every now and then (at least that is my impression).
But if you (or someone else) would like to make this a continuous project, feel free to do so. The code is attached.

Is this patch scheduled to find its way into the next release (1.2.x)?

Is it possible to import files via linux shell?

I dont want to login as admin to import csv files but would upload it via ftp to mantis temp directory. This file should get imported by a cronjob/shellscribt.

It is not a patch but a plugin and it is available for 1.2
There is a xml import/export which perhaps can be scheduled.
This plugin does not support it.

@cas

In this csv import plugin, while importing issues a default category named 'csv_imported' is created and if the category field is given empty or if it is missed out, the default category created is assigned. Instead it would be nice to have a validation if the category field is left empty while importing issues.

As for now it looks odd to have a category named 'csv_imported' in the category table as it does not mean anything.

Please let me know your suggestions in this.

@prabhurangan

at the time i've (re)written the plugin for version 1.2.x it was impossible to
import bugs without a category.
the project i have had consisted of a very large csv so i was to lazy to look through all the listed bugs in this csv and set the categroy where left blank.

also it was too much to collect it by the time of the import and then give an error to the importer, because then he/she would have had to fix this by hand.
the goal was to import the issues and the user(s) which was/were assigned to the bug(s) has/have to collect und correct the the empty fields by him-/herself.

so i've decided to write a fallback to make it possible to import issues without category and collect them under "csv_imported" so the users have a place were they can look for their bugs and assign them correct.

best regards
uso

@uso

Ya i could understand. As 'Category' is a mandatory field in 1.2.x while creating issues.

@cas

I have a big issue, in DueDate Reminder notification: It is working fine in windows; however if set it as a cron job in Liniux machine, it is not working. Also i am not receiving emails. When i checked it, we will be giving a syntax like "/1440 * lynx --dump http://mantis.homepage.com/plugin.php?page=Reminder/bug_reminder_mail.php" and in this "lynx" is a text based browser and I installed it in linux machine, even now i am not receiving any emails. If I execute the cron job, it shows me the "login_page.php" It shows the text content in that page. But the notifications is not coming.

Please let me know your thoughts....

You should really post under the correct issue. This is not about the reminder plugin.
The problem looks like an authorisation issue. I have no linux config available so cannot check.

great job for this plugin!!!
I've uploaded a german translation for the strings.

Great plugin! But it doesnt work for me. I have a problem with importing a csv. Probably i used a wrong tabledefinition. does it exist a example-csv?
thx,

configure your mantis as you want (custom fields, etc.),
adding some test case users, bogus bugs and so on...

if you've done this, use native mantis EXport capabilities to
export your db to a csv.
(dont forget to make the CSV columns fit to your needs.
=> http://<your_mantis_sleeps_here>/adm_config_report.php
or even better
=> http://<your_mantis_eats_here>/manage_config_columns_page.php)

then open the csv and see ... .

best regards

Uploaded csv_import-1.0a_mantis1.1.tgz. Tested against Mantis 1.1.7

Changes:

• For some reason all dates were being converted to epoch when they need to be mysql formatted dates (removed all "strtotime" calls).
• Comment #0018627
• Typo on line 83 of import_issues.php

Changes were made to csv_import-1.0.tgz which I believe is the last version compatible with Mantis 1.1.

I made a mistake in "csv_import-1.0a_mantis1.1.tgz" so do not download. If somebody can delete, please do so.

"csv_import-1.0b_mantis1.1.tgz" is the replacement.

using csv_import-1.0b_mantis1.1.tgz

the column detection is case-sensitive, i.e. if you have "id" instead of "ID" no columns will be detected

sorry.. but could someone provide an example csv? thanks..

I get Issue 0 not found when I try to import. I am using Mantis 1.2.0 on a fresh install.

Hi all, i'm experiencing the same problem posted by Fyb3roptik : csv file correctlyt read and values displayed, however, starting rows importation in to the database, data importing stops immediately displaying following error message :
"Message header = APPLICATION ERROR #1100"
"Message body = Issue 0 not found"
i'm trying to find out why;
my environment is :
Mantis version : 1.2.0
Schema version : 182
Importing patch version : thus belonging to uploaded folder : csv_import-1.0b_mantis1.1
Thanks in advance !
Roberto

It is rather confusing, but csv_import-1.0b_mantis1.1 is only for Mantis 1.1.x. You will have to read through all of the comments to determine which versions are compatible with 1.2.x.

So which version is compatible with mantisbt 1.2.0 and which with 1.2.1?

I'd like to import about 200 issues from a excel spread sheet. I tried the xml import with no success. Even a reimport (xml export -> xml import) does not work (bug 0010794) and does not support custom fields (bug 0011876).

This plugin is my last hope. Or are there other ways (not manually) to import a bulk of issues?

To get csv import to work under 1.2.x, what is the recommended way forward?

• Port this to a plugin e.g plugins/CsvImportExport
• Should one extend plugins/XmlImportExport for more formats similar to the import/export plugin found from http://deboutv.free.fr/mantis/
• ?

csv_import110.zip is for mantis 1.2.x

Installed csv_import110.zip as plugin in fresh install on Mantis 1.2.0, schema 183 and I'm also getting the error "APPLICATION ERROR #1100 Issue 0 not found" whenever I try to import.

My csv file is uploaded, the headers are automatically recognised and the drop-downs set to the corresponding columns but clicking import throws that error.

Anyone got any ideas what causes this?

Found that I had version 1.11 on the shelf.
Please verify, if it gives problems, please attaced sample csv file to this issue so I can investigate.

@cas

Found a couple of issues with this: The "APPLICATION ERROR #1100 Issue 0 not found" is being triggered on line 261 of import_issues.php - when $t_default->description is used, the BugData class attempts to get the description from the DB but it doesn't have an id, since you're just using it to store default values. Setting a default description around line 220 avoids this issue.

However, after doing this I'm getting Fatal error: Call to undefined function bug_create() on line 316

I can also see a call to bug_update() which doesn't exist either.

send me the csv (or attach here) you use and I will fix the issues

attached version 1.1.2 which handles reported issues.
I also found that in your importfile some mandatory fields were not filled. That should be avoided since the import program will stop in those case.
I have adjusted the importfile and saved again as csv (this time with semicolon as eperator). Next step clearly is to connect the columns to the proper fields.
Finally it imported nicely.

Thanks for your cvs_import.
Everything is good but exisiting issues cannot be updated.
Issues always are created.
The cvs_import checks id columns well.
Could you help me?
(Mantis: Mantis 1.1.0rc1, csv_import-1.0b_mantis1.1.tgz)
cvs file:
ID,Category,Status,Reproducibility,Severity,Summary,Description,Additional Information,Due Date,View Status
114,SW,acknowledged,always,major,DB1 SRS Driver I/F ? ?td>,DB1 SRS Chap 4.3 HW I/F Driver I/F ? ???.,,,public

I found the solution to update the existing issues.
The CSV file must have all columns and order.
If you entered the exisiting issue id, the issue is updated.
ex)
Reporter (ID),Summary,Description,Steps To Reproduce,Additional Information,Category,Priority,Severity,Reproducibility,Date Submitted,Last Update,Assigned To (ID),Status,Resolution,OS,OS (Build),Platform,Version,Projection,ETA,Fixed in Version,Target Version,Build,View Status,ID
,DB1 SRS Driver I/F ? ?td>,DB1 SRS Chap 4.3 HW I/F Driver I/F ? ???.,,,SW,normal,minor,always,2010-06-08 15:19,2010-06-09 09:13,,acknowledged,open,,,,,none,none,,,,public,114

The solution to updating existing issues does not work for me. I export a CSV to one file, make a minor change to the Excel file, export a second CSV file. I import file 1 no problem. Import file 2 and they're all new issues. No changes to the 'id' numbers at all.

I'm trying to mess around too see what's what, and running into stuff like the "$t_bug_exists" variable coming back as 0 (my 'id' column is column 0 in the array) which evaluates to false when simply doing a boolean check on the varialbe (it as it does 4 times in the import_issues.php file). So I replaced the "true" condition to be ($t_bug_exists >= 0). No go :(

Unless anyone else has done it, I'm also trying to create a Mapping Save feature. I've created a separate table with 4 columns (map_id, map_prj, map_cvs, map_mantis) to store the field mappings (saved per project). I'm doing this by trying to add a few new methods in the bug_api.php -> BugData Class.

I'm sure there's a better place to store these by extending the class in another file, but don't know enough about classes. The new methods I'm creating are; create_map_def(), update_map_def(), and get_map_def(). Hoping that on the mapping page, I can default in any previously selected fields from the dropdowns so that subsequent runs won't require selecting them all again.

Any input appreciated :)

Just as an FYI to anyone who didn't already know ('cause it took me awhile to figure it out), this plugin, or at least the latest version of this plugin (112), is not compatible with Mantis 1.0.8, which is the version we're using in our office.

I'm not sure which version of Mantis = which version of this plugin, but I do know that much. Just wanted to clear that up for anyone new coming into this.

UPDATE:

Just came across (or noticed rather):
$this->requires = array('MantisCore' => '1.2.0',);

The plugin system is only available as of version 1.2 so that also indicates the compatibility of the plugin.
For older versions, you need to use an earlier version (ie the last vesion before it becamne a plugin). Have a go at mantis_import_issues_v7

I tried this plug-in today, and after a few trials and errors, managed to successfully import a bunch of issues.

Many thanks to all those who contributed to this !

If I may make a suggestion, it would be nice if the imported records, would be marked as such in the Issue History, i.e. the Change Column showing the name of the file as in the example below, or something similar.

Date Modified Username Field Change
2010-09-14 19:07 Administrator New Issue Imported from xyz.csv

I just noticed that when importing issues into a sub-project which is inheriting Categories from its parent, the plug-in seems unable to match the value in the CSV's column with the ID in mantis_category_table.

In Mantis, assume the following categories exist for sub-project:
[ParentProject] Parent Category
Child Category

In CSV, I tried the following values in the Category column, without success on import (i.e. always ending up with category = csv_imported)
a) Parent Category
b) [ParentProject] Parent Category
c) <ID of [ParentProject] Parent Category>

I would expect the plug-in to be able to find a match with a).

Note: I have not tested with global categories as I do not use them currently

This is on Mantis 1.2.1 by the way.

I tried to import a csv file. Mapped the fields. At the end of the import i get the following error:

Invalid e-mail address

But there is no email address mapped.

Do you have by chance a mandatory customfield defined that would hold an email address?

Hi,
I don't understand the regexp of read_csv_row function.
Why does it care about double quotes?

If I have a string like : abcd"fzefz"feizjfz
PREG stops matching after the letter d (and ignore everything after it).

Shouldn't be like that :
/\G(?:\A|_SEPARATOR)((?!")[^_SEPARATOR]+|(?:"[^"]"))/sm

instead of
/\G(?:\A|_SEPARATOR_)([^"SEPARATOR]+|(?:"[^"]"))/sm

?

I added a new version of the plugin (1.1.3). I made two minor changes on it:

• Made the strings and errors "private" as they should be in a plugin.
• Added a french version of the strings.

Enjoy!

@jojow
Entschuldigung, I hadn't seen your file for german translation. I added it to the plugin and so uploaded version 1.1.4.
If you have time, please have a look at it to check that I didn't make any mistake. There also are a few new strings which are not translated (I kept english version for those as my german is not good enough) !

1.1.5 version is my contribution.

Changelog :

• Code closer of mantis coding conventions
• "all projects" categories reusable
• Can import "submitted date" with DD/MM/YYYY format
• Checkbox for alternative import because of double quotes
• Added helper_begin_long_process() for very long imports

Wouldn't it be good to have this src in git or svn somewhere?
e.g mantisforge

in latest 1.1.5 the version number was 1.1.3 in Csv_import.php
$this->version = '1.1.3';

Sorry for the mistake ...

Great idea !
But who should create it ?

1.2.0 version is my new contribution.

Changelog :

• import_issues_page.php splitted to become import_issues_page_init.php and import_issues_page_col_set.php
• Can create unknown categories
• Little JS to use tab as separator

open_basedir restriction in effect. File() is not within the allowed path(s)

I am getting an open_basedir restriction. However can not figure out which path should be defined.

I define the $HOME of mantis installation, the upload path for the project, the general upload path from the config_inc.php.

But none of these are OK. Which path should be added?

Do you have this statement in your config_inc.php:
$g_path = 'http://path_to_your_mantis_installation/';

Hi Cas,

not initially. ( Using 1.2.3 ). But even when I put the absolute path in, same error occurs.

But l other default stuff of Mantis regarding uploading files is working. To which (sub)directory the functions tempnam() and move_uploaded_file() referring?

SYSTEM WARNING: tempnam() [function.tempnam]: open_basedir restriction in effect. File() is not within the allowed path(s): (/home/proguide/:/home/proguide/domains/proguideweb.nl/public_html/abs-pad/upload:/var/www/ctstestdocs:/var/www/testtrackdocs:/var/www/testwiki:/var/www/devtrackdocs:/var/www/devwiki:/var/www/html/jpgraph-3.0.6:/usr/local/lib/php/:/tmp:/usr/share/fonts/)

SYSTEM WARNING: move_uploaded_file() [function.move-uploaded-file]: open_basedir restriction in effect. File() is not within the allowed path(s): (/home/proguide/:/home/proguide/domains/proguideweb.nl/public_html/abs-pad/upload:/var/www/ctstestdocs:/var/www/testtrackdocs:/var/www/testwiki:/var/www/devtrackdocs:/var/www/devwiki:/var/www/html/jpgraph-3.0.6:/usr/local/lib/php/:/tmp:/usr/share/fonts/)

Problem is in line 112 from
import_issues_page_col_set.php

Changed it into
$t_file_name = tempnam( '.', 'tmp' );

Function does not seem to like an empty directory a first parameter. Can someone perhaps comment on that?

How can you import in such a way that the due date is empty?

Default the due date is becoming the dat of today with time 00:00, this effectivly means that the item is after import overdue.

Importing with column due date set to 1, will import it as due date set to 0, which lead to displaying it as 01/01/1970.

The due dat should have the value 1, not 0 ( zero ) to display it as empty. But how can I import the value '1'?

OK, about the due date, also figured that out.

I changed the default.

$t_default->due_date = date('Y-m-d');

    $t_default->due_date = 1;

Perhaps it would be nice to make this configurable. Otherise make the default = 1, in my opinion this is more logical than assign a due date which is already overdue.

I created a github repository here : https://github.com/lionheart33806/Csv-import-4-MantisBT

Thanks for that, lionheart33806!
Shall we start using the issues section instead of this bug, which seems quite inappropriate and hardly usable?

Sounds like a great idea ! This page is totally unmanageable.

I'm agree to use issues section too.

I've just release v1.3.0b on github (if you don't use git, just use "Downloads" button).
Please test it.

Lionheart,

could you please comment on my remarks:

0004220:0027395, 0004220:0027396 and 0004220:0027397

Ok, try this source : https://github.com/lionheart33806/Csv-import-4-MantisBT/commit/515e2a299f2a2f55a07b6617e35fd798f3b075c0 (via "Downloads")

Please next time use "Issues" section of github

lionheart33806, are you aware that you can have an own project for this plugin at this instance of MantisBT tracker?
Have a look at the project selection box, where user cas is the manager for the four plugin's

Sorry my english is bad, i don't understand you ...
Are you saying that I shouldn't create a githit repo ?

I think the repo is Ok, but a better place for this would be http://git.mantisforge.org

I was talking about issue tracking for the plugin.
This could be handled with an own project at www.mantisbt.org/bugs

Have a look at the project selection list, where you will find that there are projects for four plugins at the momement.

I get it now, thanks !
I was looking for this before creating in github but I didn't found it ...

I'll look it when I'll have more time !

THANK for moving you code to http://git.mantisforge.org !

Just discovered your plug-in, need to try it a little more however found one glitch

The strings with "Csv" should be "csv" , mind the capital "C"

$s_plugin_Csv_import_title = 'Mantis CSV Importer';
$s_plugin_Csv_import_description = 'Import CSV files as Mantis issues';

When I've added them to custom_strings_inc.php it worked.

$s_plugin_csv_import_title = 'Mantis CSV Importer';
$s_plugin_csv_import_description = 'Import CSV files as Mantis issues';

Where under http://git.mantisforge.org have you added this plugin, I tried looking for any plugin with cvs or import there but could not find any.

We use this plugin to export/import issues beween ms project and I need to fix a couple of things from the version found 0004220:0027475

-When exporting CVS the files comes in UTF-8 but import expect e.g Latin1
-Export takes Real Name but import expect username

So the question is these fixed alreday if yes in wich repos as I seams not be be able to find it under http://git.mantisforge.org and if no where should I commit the changes if I fix these?

Nice plugin!
Would like to make some suggestions though:

• when selecting multiple times 'ignore_column' the script failes due to the check for unique columns
• first checking if the required fields are filled prevents importing half the list (e.g. 'description')

Get the latest version via Git https://github.com/lionheart33806/Csv-import-4-MantisBT

Fork it, change the defects, and ask the maintainer, to do a Pull request to get all fixes into the mater original branch.

You could then delete your fork

I was not able to use this plugin. In the readme, the installation part says
"Installation instructions are the same as installing a plugin."

I suggest to describe the full procedure installation in the readme because it's not easy to find the procedure on the net (essentially found instructions for old mantis versions with pluginmanager that is not compatible with new mantis any more).

thank you.

@lionheart:
Would you please host the plug-in to the now official plug-in place for Mantis which is:
http://github.com/mantisbt-plugins/
Please contact John Reese (who has just created this organization) on the developer mailing list if you need help on this.
A specific project for this plug-in should also be created in this Mantis as it is the case for some other plug-ins.
I think that once done, this issue, which is getting longer and longer with all those comments, will have to be closed.

Is the repos https://github.com/lionheart33806/Csv-import-4-MantisBT alive any more? Created a little patch for the mantis Csv plugin that also can take real names and not only user names and if the file contains a UTF-8 BOM characters it will expect the file to in utf-8 instead of Latin-1

I sent this patch file to lionheart33806 19.6.2011 but it seems to have gone out in space. However I have attached the patch here 0001-Utf8-support-and-Real-names.patch

Hi Guys, I've found this plugin useful and I'm about to use it on our production environment. I have tested it on my local and it seems fine. However, it seems that it doesn't send emails for the uploaded entries. Do you think if it is possible that it can be tweaked to send emails for all the entries imported by this plugin? Just the same as submitting a single entry.

Sending emails for each of the bugs imported through this plugin.

Guys, I was able to fix what I need and I just want to share this piece of idea. Actually , this is based on the bug_report.php included in the mantis core package. If you want to have an automated email sent for each of the new bugs imported by this plugin, just add these lines from bug_report.php to import_issue.php included in this plugin just before the # Result comment:

helper_call_custom_function( 'issue_create_notify', array( $t_bug_id ) );
email_new_bug( $t_bug_id );

Result

if($t_error) {
$t_failure_count++;
}
else {
$t_success_count++;
}

Hope it helps!

I'm (a little) back !

Sorry, I didn't have time to work on the plugin.

Let me a bit of time to see your notes.

@Rustan : Interesting idea ! I added it in github "issues".

@andy778 : I can't find something (mail or github) about your patch :-/

@sveyret & Georges_ : It's on the way.

@cor3huis : Is your plugin folder named "csv_import" ? Try "Csv_import".

@AboeBakr : Can you be more "clear" about the 2nd problem ?

@lionheart33806 the patch is also here as an attachment 0001-Utf8-support-and-Real-names.patch

Guys, I encountered issue when trying to test in on our dev environment:

SYSTEM WARNING: file_exists() [function.file-exists]: open_basedir restriction in effect. File(/tmp/D9R8wM) is not within the allowed path(s): /data/otherlocation

Would you know where can I tweak this location? I think it points by default to /tmp location and it is not where we upload temp files.

Your help is very much appreciated.

Look at sys_get_temp_dir() (import_issues_inc.php, import_issues_page_col_set.php and http://php.net/manual/en/function.sys-get-temp-dir.php)

@lionheart33806 - it does not get inside the if condition:

import_issues_inc.php:

if ( !function_exists('sys_get_temp_dir')) {
function sys_get_temp_dir() {
if( $temp = getenv('TMP') ) {
return $temp;
}
if( $temp = getenv('TEMP') ) {
return $temp;
}
if( $temp = getenv('TMPDIR') ) {
return $temp;
}
$temp = tempnam(FILE,'');
echo "\$temp: ".$temp."
";
if (file_exists($temp)) {
unlink($temp);
return dirname($temp);
}
return null;
}
}
echo "\$temp: ".$temp."
";
echo realpath(sys_get_temp_dir());

The output of this one is:

$temp:
/tmp

Can i override the system temp directory to point to /data/otherlocation?

@lionheart33806

regarding the second problem,
some fields, for example 'description' are required.
When the csv has a row, where the column mapped to 'description' is empty,
the import will fail.

the problem is: it inserts all the entries up to the row with the missing description.

as it is now, you have too look up at which point in the csv it goes amiss,
then remove all the rows above this point, save the csv and then restart the import.

the line at which it is failing is not reported as well, making this even more inconvenient.

If the csv would be scanned for these missing fields, you could either import all entries or none. And preferably report to the user at which row/column there is a problem.

@Rustan
sys_get_temp_dir() as the name indicates, returns the system temp dir (which is not the same as the upload temp dir). I assume that you expect the plugin to use the location specified in upload_tmp_dir, but that won't work (see https://bugs.php.net/bug.php?id=54710).

To work around the problem, assuming you're using apache, you could try to

• modify your apache envvars file, to add: export TMPDIR=/data/otherlocation
• modify your open_basedir to allow your system temp dir (there could be security implications there though), e.g. /data/otherlocation:/tmp

Maybe the plugin should be improved to check that the path returned by sys_get_temp_dir() is checked for accessibility prior to further processing.

The repository moved to https://github.com/mantisbt-plugins/csv-import

@Rustan : please try latest tarball

@dregad - thanks for the info, actually, i understand that it uses the system-defined temp dir but, as i have said, we have a defined temp location to write on such as /data/otherlocation. your suggestion can be done on the server side. can i change the location, say define the /data/otherlocation because i know that where i have write privilege?

@lionheart33806 - thanks, ill try that.

My changes only try to prevent the warning message.

@lionheart33806 - thanks, it works. now, when i implemented this into our production, im having problem when i clicked the upload file button, here is the error:

APPLICATION ERROR #11
A necessary field "" was empty. Please recheck your inputs.

Please use the "Back" button in your web browser to return to the previous page. There you can correct whatever problems were identified in this error or select another action. You can also click an option from the menu bar to go directly to a new section.

The CsvImport project is now created in Mantis BT (see the project drop down at top right of this window). If you have any more problems concerning this plug-in, please open a new issue in the corresponding project.
Can someone close this issue to prevent people from adding information?
Thank you.

I am trying to import a csv file using the [ Import CSV file ] plugin link in the manage screen. I installed Csv_import.120.tar.gz

I get the same error like m.vanhamme above: "Invalid e-mail address". There is no e-mail address in the csv file, in fact there is not even an @ sign. The source is actually a converted xml file from a mantisbt export, only with default fields. Fields as defined in the first line of the csv: ID, Reporter, Assigned To, Priority, Severity, Reproducibility, Product Version, Category, Date Submitted, OS, OS Version, Platform, View Status, Updated, Summary, Status, Resolution, Fixed in Version. (no space after comma in csv, added here for clarity)

Suspecting that the converter is choking on certain content I tried to convert only the first 10 records of a 480 record csv file. Same result.

Then I tried to import records 10-20. Now I get this: "APPLICATION ERROR #11. A necessary field "Description" was empty. Please recheck your inputs." There is no Description field?

I also tried to import the xml using the [ Import issues ] plugin link. This fails showing an empty screen with URL: https://example.com/mantisbt/plugin.php?page=XmlImportExport/import_action - an no data imported.

Any help in how to import data from another mantisbt is greatly appreciated, thank you.

Celebrity alert ! We've got the famous creator of TWiki in the house :-)

Hi Peter,

I can't really help you with your question, hopefully lionheart33806 who maintains the plugin will have an answer for you. Nevertheless, I would suggest to get the latest source for the plug-in from Github [1], instead of the tarballs attached to this issue.

Damien

[1] https://github.com/mantisbt-plugins/csv-import

Thank you Damien. I installed the latest Mantis CSV Importer 1.3.0b from github.

Now I get this error:

APPLICATION WARNING #300: String "plugin_Csv_import_result_nothing_success_ct" not found.

When I omit the ID field I get again this error:

APPLICATION ERROR 0001200: Invalid e-mail address.

Hmm...

Apparently the Description field is mandatory, so I added it. However, import still gave me the e-mail error.

I did a brute force import: Wrote a script that uses curl to push the data record by record via http posts. Problem solved (with much more effort than it should be).

Thanks for your help anyway!

about 0004220:0030141 , if you try to import handler or reporter who doesn't exist in your mantisbt, then it will try to create it...

but function user_create() in user_api.php do "email_ensure_valid(email)" even if email is '', that's an "Invalid e-mail address."

so, don't import handler or reporter or write in plugins\csv_import\pages\import_issues_inc.php :
line 253:
if( user_create( $t_username , $t_username, 'nomail@for.me' ) ) {

Not good for security : password = username...

teste

VeMag is correct. If you are trying to assign an entry to a user with invalid email address, it would give you an error of invalid email address. to fix this, you can disable the checking of valid email addresses of users being assigned to an entry.

hI, FUNNY PROBLEM!

When I import a CSV file everything goes well.
It generates new issues and all information for the standard mantis fields are filled in.

Except for the custom fields.
It reconizes the customfields, I can sellect them in the import tool.
But it just does not import the information for the custom fields.

Is there something I forgot?
Do I have to add somewere the custom fields I want to import?

I've latest verion 1.3.0b, mantis 1.2.8.

Resolving this issue in favor of the version of the extension hosted in github and has been getting the recent fixes.
https://github.com/mantisbt-plugins/csv-import/