Oops... Mantis messed a little...

GraphViz URL is http://www.graphviz.org/

The first two files had their names chopped... originally their were graph-example.png and relationship-graphs-patch.diff.gz... another issue? The contents seems to be ok.

Looks good (at first sight)! Could you make a couple of changes to the code:

• Move the functions appropriately named into a core/..._api.php file
• Write lines as

if ( $f_return == "graph" ) {

like

if ( 'graph' == $f_return ) {

• Perhaps there is an advantage in using the Image_GraphViz PEAR package. Did you take a look at it?
• It might be cleaner to just move the image generation script in a separate file (and the text output, but I can't tell what the use of that is - debug?)

I take it this only works on UNIX systems?

As for the name chopping... I thought we'd got rid of that. File an issue about it?

Ok, I'll do these changes and upload them soon. I didn't know that PEAR already had a class for GraphViz... just take a quick look at the documentation and it seems to be very nice.

The text output was really used for debugging, and since it is quite harmless, I decided to leave it there to avoid reimplementing it if it was going to be used again in the future. If you think it is not appropriate to leave it, I'll enclose it in a comment (but I think it will no longer be necessary using Image_GraphViz).

I'm not a Windows user, but I know for sure that there is a GraphViz package for Windows, wait... found, it is here: http://www.research.att.com/sw/tools/graphviz/download.html . Someone will have to volunteer to test it under Windows when I release my next patch.

About the name chopping, I filled the bug 0004437.

Done.

Now there are two interface files: bug_relationship_graph.php, which serves the page, and bug_relationship_graph_img.php, which returns the graph in png format. The whole algorithm was moved to core/relationship_graphapi.php, under the "relgraph" namespace. Inline source documentation was greatly improved.

Now it uses PEAR Image_GraphWiz module to generate the graph, like suggested. It is not really what I was expecting, but it does the job. I have included the latest Image_GraphViz module as core/class.GraphViz.inc.php (check if the naming/placement is good). It requires PEAR's System.php, which is not included since it should be present on every PHP instalation (at least I think).

The files in (relationship-)graphs-patch.diff.gz were not changed.

OK, what about changed small HowTo for kamikaze, which want to try it now? Which changes in config_inc.php may be "obsolete" (if any?)

Wanderer: Huh?!

I want to try graphs it on my site - "What I (or anybody) have to do, except adding files and editing config?"
I know nothing (well, "lets suppose, I know nothing...") about GraphViz and want to get such nice images in BT for demonstrating it to boss. What I must require to do company's admin?

Oh, I see now... I was just wondering what you did mean by kamikaze, howto and obsolete config variables... :-P

You didn't say your operating system, so I'll give generic instructions.

Firstly, apply the (relationship-)graphs-patch.diff.gz (it adds some config vars to config_defaults_inc.php, does a very small addition to core/relationship_api.php and updates language files (en and pt-BR)). Install the files in relatiohship-graphs-files.tar.gz (bug_relationship_graph.php, bug_relationship_graph_img.php, core/class.GraphViz.inc.php and core/relationship_graph_api.php). These are the only changes necessary to mantis itself.

Then, you need to install GraphViz tools. They are available at http://www.research.att.com/sw/tools/graphviz/download.html , but if you are using a Linux distribution, check it because it probably already has a package (for Redhat and Fedora you must get if from the same site). For Windows, you have the option of only installing the dot.exe utility (that is everything you need).

Once it is installed, open your config_inc.php and add:

$g_relationship_graph_enable       = ON;

$g_relationship_graph_fontname      = 'Arial';

This will enable the feature. You should set a font that is accepted by dot, this depends on your operating system and how the dot utility was compiled. Windows is simple, just the name of the font you want to use. For linux, try "Bitstream Vera Sans", "luxisr" and "helvetica". If none of these work for you, you may have to tweak it a bit.

Finally, this config:

$g_dot_tool   = '/usr/local/bin/dot';

$g_dot_tool   = 'C:\path-to-graphviz\dot.exe';

to tell where the dot tool is located.

Once you view an issue in the bugtracker, you'll note a "[ Graph ]" link on the top of the expanded Relationships box. Just click it and enjoy!

First, I would like to thank jferraz for such a valuable contribution. Good work.

I agree with jlatour's comments. However, I am not sure of how much benefit to do we get from using PEAR's GraphViz class. The issues I am seeing are:

1. This class depends on other PEAR clases (eg: System, and System depends on getOpt, ...etc).
2. I think that the minimum PHP version requirement for PEAR is higher than Mantis. On the other hand, I think we can safely higher our PHP minimum requirements.
3. Given that PEAR is not used in the rest of Mantis, I think we should not just add it for one feature (specially if we don't have to). If we decide to add a dependency on PEAR, then we should probably use it in a lot of other places in the code.

Will the code be much more complicated with the PEAR class? Is there a GPL non-PEAR alternative for wrapping the "dot" tool?

Forgot to ask about how do we know the relationship "type" from the graph? i.e. related to, parent of, ...etc?

I'm not saying we should use it, just asking if there are advantages to it.

Hi Victor,

I'm happy that you liked it! It is a pleasure for me to help Mantis development, since it really fixed the issues (lol) I had with bugzilla and anthill! Too bad I don't have more free time to contribute regularly.

About GraphViz modules, I must confess that I was quite disapointed with it. I have already used some PEAR classes before (mostly the core) and this was my first deception with any of them. This was one of those software that you use the first time and want to recreate it yourself the right way. Not that GraphViz PEAR module is that bad, but a regular dotter will miss some very handy features of the dot language.

The current implementation is using it, but it will be fine for me to revert it to hand-generating the graph (or even by implementing our own dot wrapper class). I don't know of any other wrappers like this one. The code using the GraphViz class is much more cleaner, but we can still reimplement it without depending on PEAR (and without the issues I have found).

Currently, it only shows "parent-child" (dependency) relations (this is what I really needed as a developer). There are some limitations with other relationship types, imposed by GraphViz. Explaining:

Dot generates directed graphs, this is, the nodes are ranked, flowing mostly in only one direction. If you look at the example, you will see that parents are to the left and children to the right, dot does this automatically. For "related to" relations, there is no concept (not at the user level) of source and destination, they are undirected. For "related to" relations, neato (another GraphViz tool) is more appropriate. Unfortunately, dot is very good with directed graphs and neato is very good with undirected graphs; there is no middle way.

But we still have the possibility to have different graphs for "parent-child" relations (using dot) and for "related to" relations (using neato), and even a third type for "duplicate of-duplicates" relations (using dot).

I just had an idea to force ranking of nodes and/or using subgraphs to create a mixed "related to/parent-child" graph. I'll try it later to see how it looks like. If I manage to mix "related to" and "parent-child" relations on the same graph, it is really simple to change the style or the color of the edges.

OK, we're awaiting your results!

The third version... Enjoy!

relationship-graphs-3-patch.diff.gz (1,741 bytes) 09-01-04 18:31

relationship-graphs-3-files.tar.gz  (8,803 bytes) 09-01-04 18:31

rather then using an external application, what about just using a php graphing module for this?

I really like the concept, however, i think an php solution would be best. Or at least, as a fall back...

I seem to recall seeing php graphing classes to do graphs like these in the past

Grangeway:

Are you sure that it really exists? I had googled a lot, visited many php sites and the only ones I had found were interfaces to the GraphViz tools. It should be quite difficult to do this using only php, since they are complex (specially for neato) algorithms.

FYI, my compiled libdotneato.so is almost 1MB big, and this doesn't count other graphviz internal libraries that this one depends on. Imagine this ported to php...

And this is still an option to the Mantis user, defaulted to OFF.

I only had a quick look last night at both your routine, graphviz and for graphing classes for php. There are some classes that would take a structured array and display it as a graph. Of course, and as you quite rightly stated, to calculate whether a node is a duplicate of another node, and also to display a best shaped graph is a lot more difficult. i.e. The graph in your example could easily be represented by php -> it's a simple tree structure.

Where php would fall over completely is if you had say, 100 nodes on the diagram.

I personally use mantis under windows. I seem to remember from months ago, that for php under windows to run an application e.g. graphviz, it would need execute permissions on cmd.exe (something which you wouldn't want) - ths might have changed in the last 18 months, so i'd need to revisit it.

That's also my main basis for being keen on a php version (which could be increased to use graphviz instead for increased performance / functionality etc).

Seems that I need to go and do some testing on my various windows boxes. And as i neglected to say it in my previous note, jferraz - stunning work.

Maybe we could do a php implementation that shows 1? 2 levels or some fixed amount.. (with a note that suggests users 'extend' to use graphviz.)

Grangeway, thank you very much.

You still missed to say which classes are those... :-P

It is strange this limitation on cmd.exe. My implementation uses proc_open() if php version is >= 4.3.0 or passthru() otherwise. Both, AFAIK, don't rely on a shell; which is differs from shell_exec() command that, like the name says, uses the shell to execute the command. Maybe the windows php uses a different implementation.

I don't know if you noticed, but the latest version has two types of graphs: a dependency graph (directed, using dot) and a relationship graph (undirected, using neato). Since dot is faster and simpler, dependency graphs are drawn to their full depth; on the other side, neato has some sensible overhead with really large graphs, and so, relationthip graphs are limited to $g_relationship_graph_max_depth levels.

Please, when you do it, tell me how it worked on your windows boxes, I'm really curious. And tell me where did you see this php class that generates (un)directed graphs so I can check if it is feasible to support it.

I attached an example of a relationship graph, available on the third version of my patch. Note that only issues not more than 2 relationships far from 0000075 were shown. You can configure this using $g_relationship_graph_max_depth.

For 'simple' graphs, there's something on phpclasses that generates E-R type graphs i.e. a set of boxes with lines indicating various values (e.g. in our case could be child/parent). There's also classes that do implement 'tree structure's

I was initially focusing more on your original graph thinking, that most graphs will be quite simple. i.e. a single parent, 3 children + 3 * 3 children for example -> obviously in some cases it could be a lot more complex then this!!.

jferraz ,
Php prepends "cmd.exe /c " in proc_open/shell etc prior to running an external script.

In windows 2003, by default, IIS does not have access to cmd.exe

relevant lines of PHP source:
command_with_cmd = emalloc(command_len + sizeof(COMSPEC_9X) + 1 + sizeof(" /c "));
sprintf(command_with_cmd, "%s /c %s", GetVersion() < 0x80000000 ? COMSPEC_NT : COMSPEC_9X, command);
newprocok = CreateProcess(NULL, command_with_cmd, &security, &security, TRUE, NORMAL_PRIORITY_CLASS, env.envp, cwd, &si, &pi);
efree(command_with_cmd);

edited on: 09-15-04 15:59

Jferraz,

Is the current attached patch your 'latest' atm?

Paul

Grangeway,

I'll try to find the class that you refer to and see if it is possible to port my patch to it.

Man, I think this is a real issue with PHP... I didn't understand why the hell PHP calls CMD to execute the command. Perhaps to allow users to use a internal shell command, like "dir"? But this sure is reserved to shell_exec(), and neither passthru() nor proc_open().

Surely IIS must never have access to cmd.exe... never. This is really disappointing... I made this in the hope to help all mantis users, and not restrict it to linux-only... :-/

Yes, the latest is the two files with version "3" marked in the filename.

jferraz: I just have to say....this is AWESOME! This and the relationships together have really pushed this application to the next step. Congrats and great job so far.

Jferraz,

I'm currently thinking the following:
a) this patch would add value to users of mantis
b) for windows users providing the options of:
i) windows cmd /c execution
ii) com object execution (i.e. via wingraphwiz).
.. might be beneficial.

Part 'b/ii' i might take a look at.

Paul

Virtuetek: Thank you! It is really rewarding to see that the community liked it so much. :-)

Grangeway: I don't think that cmd /c is an option under windows... allowing either IIS or Apache execute cmd.exe is the same as allowing it to execute anything. Just the idea scares me. About using WinGraphviz through COM, it seems to be a very good idea!

I'm going take a look at it, but since I'm not a Windows programmer, I'll need some time to study COM and learn how to use its API provided by PHP (unless you want to take it). The only file that needs change is graphviz_api.php.

By looking at PHP manual, it seems that COM is supported since PHP 3, this is good, but the former API is deprecated in favor of the COM class, which version information unfortunately isn't part of PHP manual... do you know if it was available in PHP 4.0.6 (minimum req of Mantis)?

COM support was rewritten between PHP 4 and PHP 5.

in PHP4, it was /ext/com
in PHP5, initially someone was working on a /ext/rpc module (which would provide dotnet/com/java support)
It seems that since then, they've moved that to pecl for now, and implemented a /ext/dotnet_com version, which is mainly compatible with /ext/com from php4 but they've dropped some features.

It should be relatively easy to use the COM version + your code.

graphviz for windows is available from http://home.so-net.net.tw/oodtsen/wingraphviz/index_en.htm.
Download the zip, regsvr32 the dll file, and then some code similar to i think would work:

$graphviz = new COM('Wingraphviz.neato');
$ret = $graphviz->ToPNG(ob_get_contents());
$graphviz->release();
$graphviz = null;

The forth version!

jferraz-relationship-graphs-4-patch.diff.gz (1,812 bytes) 09-19-04 15:10

jferraz-relationship-graphs-4-files.tar.gz  (8,201 bytes) 09-19-04 15:11

Now it supports WinGraphviz under Windows. Thanks Grangeway!

Brief installation instructions (either mantis 0.19.0 or latest cvs):

1. On Windows, install WinGraphviz from http://home.so-net.net.tw/oodtsen/wingraphviz/ ; on other platforms, install Graphviz from http://www.graphviz.org/ .


2. Extract the files contained in jferraz-relationship-graphs-4-files.tar.gz into your Mantis directory.


3. Apply the patch in jferraz-relationship-graphs-4-patch.diff.gz .


4. Add the line

   $g_relationship_graph_enable = ON;

   to your config_inc.php file (refer to config_defaults_inc.php for other configuration options).


5. Enjoy!

heh, php crashes for me at work :(

One thing that i did notice:

if the COM object isn't registered, we need to handle it

Possibly wrap around new COM ?
$foo = new COM (graphwiz);
if (!is_null($foo)) ...

Or whatever.

Btw, did you test the COM stuff yourself with no problems ?

I installed this on RH8.0 today. We need to add an additional parameter to define the 'fontpath' to the graphviz tool. The default font causes some spacing errors. This needs to be added as a global parameter, and to the generation of each graph.

(As an aside, the fonts from jpgraph work here).

Grangeway:
Huh! I'd just forgot that! :-P I'm fixing this.
I installed Apache+PHP+MySQL+Mantis+WinGraphviz on my WindowsME partition just for implementing and testing this under Windows. Some usual problems to configure MySQL but it worked really nice in the end, and I could implement the WinGraphviz support via COM just like you shown me (and by some reading of PHP manual and WinGraphviz documentation). Since it is very simple and straightforward, I think it will run nicely on IIS too, but you may still check it for me, please. :-)

Thraxisp:
Yeah, I did this. I'll upload it in a few minutes. But remember that the default font search path list is defined during graphviz compilation, if you compiled or are compiling graphviz yourself, it may be better to set it during ./configure to have it working for everything in your machine. Another option is exporting the DOTFONTPATH environment variable on your /etc/rc.d/init.d/httpd script. On this next version, you'll also have a $g_relationship_graph_fontpath. :-)

And here comes the fifth version...

jferraz-relationship-graphs-5-patch.diff.gz (2,184 bytes) 09-20-04 20:41

jferraz-relationship-graphs-5-files.tar.gz  (8,431 bytes) 09-20-04 20:41

• Fixed COM support if WinGraphviz can't be loaded;
• added $g_relationship_graph_fontpath;
• fixed minor bug with Vertical/Horizontal switch.

Jferraz
It works fine (s/fine/great!/) at home, crashes at work with a php issue. Trying to work out what's going on atm

Regarding the config variables/fonts stuff,

• $g_relationship_graph_fontpath = '';
  Can we not just use mantis/captcha font path we use atm?

• $g_dot_tool = '/usr/bin/dot';

• $g_neato_tool = '/usr/bin/neato';
  Similarly, $g_pathtographwiz apps, or is it likely that someone would have different paths/names to neato and dot tool ?

• $g_relationship_graph_view_on_click = OFF;
  I half wonder how much that one is needed - I kind of envisage if you cliked a link to a bug, you'd get graph (of that bug + view.php output for that bug on the same screen)

$g_relationship_graph_orientation = 'horizontal' seems like something that it might be nice to vary (automatically) between horizontal/vertical depending on the number of bugs (i've not actually looked in detail at the code that's generated the graphs)

One of the things I started to raise on IRC recently was about 'per-user'/project config entries and also about the number/dependencies of some config entries which can be confusing in some cases. The other thing was about possibly applying similar logic used by other projects to provide an graphical installer for the 'main' config entries. For graphviz stuff, i'd expect to see
$g_relationship_graph_enable
$g_path_to_graphviz (if null, then use com)

goes to find a debugger to hook around php

I installed graphviz from RPMs (as I expect most sysadmins will). It doesn't seem to reference fonts from anywhere specific (but I haven't looked at the code). You could also reuse the $g_system_font_folder (same truetype fonts for use in the 'captcha' graphic).

About the font path variable, i see that it defaults to 'c:/winnt/fonts/', which is quite problematic to assume for graphviz... Graphviz works nicely under windows if you don't specify a fontpath, since it knows precisely where is the system font folder. Assuming the same value for dot will restrict this behavior and add a new variable for users to configure under windows, dot will no longer look for fonts on the (presumably) right directory (for example, mine is c:/windows/fonts/).

About the path to graphviz tools, I did this way because an user could have wanted to add ".exe" extension after each name under windows (which is no longer a problem with WinGraphviz). Do you think something like $g_db_table_prefix and $gmantis*_table? This has a drawback: setting just $g_db_table_prefix does nothing useful, since all other variables were set using the value formerly loaded on config_defaults_inc.php (check 0004527). Doing something like:

$g_graphviz_app_path = '/usr/bin';

$g_dot_tool = $g_graphviz_app_path.'/dot';

$g_neato_tool = $g_graphviz_app_path.'/neato';

About view-on-click and orientation, they were added because I thought it was worth giving this ability to the user. One of the things I like most in Mantis is its customizability. About the orientation, it will be a lot tricky to add this feature, and I don't know if it is worth the work... It is used only once when the user first view a dependency graph. After that, the user may change it via the "Horizontal/Vertical" switch and will stay in the chosen mode until he/she stops navigating the relationship graphs (i.e., the orientation is passed forth via URL every time the user clicks a new issue). I think this is better than changing the orientation automatically for every issue the user visits.

When you say "per-user" configuration, do you mean user levels or just every single user? These two options are be good candidates for user account preferences, or even just caching the last mode the user used.

gah, we need a 'quote' button on bugnotes!

About the font path variable, i see that it defaults to 'c:/winnt/fonts/', >

Ignoring the current 'default', if a user was to install both relationship graphs and captcha support, wouldn't they normally need both font paths to be the same ?

About the path to graphviz tools, I did this way because an user could have
wanted to add ".exe" extension after each name under windows (which is no
longer a problem with WinGraphviz).

Good point, and i agree - remind me, if you are on windows do you have to use COM ? (i'm thinking something like 'if path to neato is blank then use COM' (or another config variable :(). I've not done any performance testing of COM vs executing a file under windows, but is it feasible someone could have access to run graphviz.exe but not register a com object?

Do you think something like $g_db_table_prefix and $gmantis_table? This has a drawback: setting just $g_db_table_prefix does nothing useful, since all
<sidenote> This is something discussed v. briefly on irc recently, about trying to tidy up for 0.20/1.0 -> in general, $gmantis_table could be 'deprecated' (or more moved from config.php to elsewhere), only keeping g_db_table_prefix for visible user use </sidenote>

When you say "per-user" configuration, do you mean user levels or just every
single user? These two options are be good candidates for user account
preferences, or even just caching the last mode the user used.

A user preference

I think I'll also assign this to me and look at getting this code checked into CVS.

As far as you are concerned jferraz, is it 'ready to go' ?

| Ignoring the current 'default', if a user was to install both relationship
| graphs and captcha support, wouldn't they normally need both font paths to
| be the same ?

Yes. Just a note because there is really no need to set a font path for dot on Windows.

| if you are on windows do you have to use COM ? (i'm thinking something
| like 'if path to neato is blank then use COM'

Yes, in fact, graphviz_api.php detects a Windows PHP and uses COM automatically. But if you look at graphviz_api.php, you'll notice that it does not rely on anything of Mantis... I did this to promote reusability. Such a test should go to relationship_graphs_api.php, and this would imply some changes to the constructurs of Graph and Digraph classes.

| but is it feasible someone could have access to run graphviz.exe but
| not register a com object?

I think that if the user is going to run Mantis with Graphviz, he/she should be an administrator, so it is not a problem to register the COM object... And I think any sane admin would choose the COM object due to its improved security.

| I think I'll also assign this to me and look at getting this code
| checked into CVS.
| As far as you are concerned jferraz, is it 'ready to go' ?

Yay! :-D Just handle the above issues the way you think will better fit Mantis phiosophy and go ahead. :-)

For reference, dot (et al) on Linux (RH) use the following compiled in default font path.

"/usr/X11R6/lib/X11/fonts/TrueType:/usr/X11R6/lib/X11/fonts/truetype:

/usr/X11R6/lib/X11/fonts/TTF:/usr/share/fonts/TrueType:/usr/share/fonts/truetype:

/usr/openwin/lib/X11/fonts/TrueType:/usr/X11R6/lib/X11/fonts/Type1"

The problem that I ran into is that the standard Windows fonts aren't installed there by default. This makes for crappy looking graphs when it uses the default internal font. Where to put the fonts should be in the docs.

Ok, your version of this is committed to CVS with a few minor (whitespace) changes.

Paul