Skip to content

How to setup OpenGrok

Jump to bottom
Vladimir Kotal edited this page Jul 27, 2018 · 80 revisions

OpenGrok can be installed and used under different use cases. Advanced usage depends on your knowledge of running java applications and command line options. Note, that you need the create the index no matter what is your use case. Without indexes Opengrok will be simply useless.

Requirements

You need the following:

After unpacking the binaries to your target directory, the index needs to be created and the web application deployed.

Creating the index

The data to be indexed should be stored in a directory called source root. Each subdirectory under this directory is called project (projects can be disabled but let's leave this detail aside for now) and usually contains checkout of a repository (or it's branch, version, ...) sources. Each project can have multiple repositories.

The concept of projects was introduced to effectively replace the need for more web applications with opengrok .war and leave you with one indexer and one web application serving more source code repositories - projects.

The index data will be created under directory called data root.

Step.0 - Setting up the sources

Source base should be available locally for OpenGrok to work efficiently. No changes are required to your source tree. If the code is under CVS or SVN, OpenGrok requires the '''checked out source''' tree under source root.

Step.1 - Deploy the web application

Install web application container of your choice (e.g. Tomcat, Glassfish).

Use the deploy.py script to copy the .war file to the location where the application container will detect it and deploy the web application.

Step.2 - Indexing

This steps basically performs these steps:

  • create index
  • let the indexer generate the configuration file
  • notify the web application that new index is available

The indexing can take a lot of time - for large code bases (meaning both amount of source code and history) it can take many hours. After this is done, indexer automatically attempts to upload newly generated configuration to the web application. Until this is done, the web application will display the old state.

The indexer can be run either using opengrok.jar directly or using the indexer.py wrapper like so:

indexer.py -J=-Djava.util.logging.config.file=/var/opengrok/logging.properties \
    -a /opengrok/dist/lib/opengrok.jar -- \
    -s /var/opengrok/src -d /var/opengrok/data -H -P -S -G \
    -W /var/opengrok/etc/configuration.xml` -U http://localhost:8080

The above will use /var/opengrok/src as source root, /var/opengrok/data as data root. The configuration will be written to /var/opengrok/etc/configuration.xml and sent to the web application (via the URL passed to the -U option) at the end of the indexing.

Run the command with -h to get more information about the options, i.e.:

indexer.py -a /opengrok/dist/lib/opengrok.jar -- -h

Optionally use --detailed together with -h to get extra detailed help, including examples.

It is assumed that any SCM commands are reachable in one of the components of the PATH environment variable (e.g. the git command for Git repositories). Likewise, this should be maintained in the environment of the user which runs the web server instance.

You should now be able to point your browser to http://YOUR_WEBAPP_SERVER:WEBAPPSRV_PORT/source to work with your fresh installation.

In some setups, it might be desirable to run the indexing (and especially mirroring) of each project in parallel in order to speed up the overall progress. See https://github.com/oracle/opengrok/wiki/Per-project-management on how this can be done.

Optional info

Custom ctags configuration

To make ctags recognize additional symbols/definitions/etc. it is possible to specify configuration file with extra configuration options for ctags.

This can be done by using the -o option for opengrok.jar.

Sample configuration file for Solaris code base:

--regex-asm=/^[ \t]*(ENTRY_NP|ENTRY|RTENTRY)+\(([a-zA-Z0-9_]+)\)/\2/f,function/
--regex-asm=/^[ \t]*ENTRY2\(([a-zA-Z0-9_]+),[ ]*([a-zA-Z0-9_]+)\)/\1/f,function/
--regex-asm=/^[ \t]*ENTRY2\(([a-zA-Z0-9_]+),[ ]*([a-zA-Z0-9_]+)\)/\2/f,function/
--regex-asm=/^[ \t]*ENTRY_NP2\(([a-zA-Z0-9_]+),[ ]*([a-zA-Z0-9_]+)\)/\1/f,function/
--regex-asm=/^[ \t]*ENTRY_NP2\(([a-zA-Z0-9_]+),[ ]*([a-zA-Z0-9_]+)\)/\2/f,function/

Introduce own mapping for an extension to analyzer

Use the -A Indexer option, e.g. like this:

-A cs:org.opensolaris.opengrok.analysis.PlainAnalyzer

This will map extension .cs to PlainAnalyzer. You should even be able to override OpenGroks analyzers using this option.

Optional need to change web application properties or name

You might need to modify the web application if you don't store the configuration file in the default location (/var/opengrok/etc/configuration.xml). This can be conveniently done using the deploy.py script by supplying the path to the configuration.

If you need to change name of the web application from source to something else, just deploy source.war as new_name.war.

Deploy the modified .war file in glassfish/Sun Java App Server

  • '''Option 1:''' Use browser and log into glassfish web administration interface

: Common Tasks / Applications / Web Applications , button '''Deploy''' and point it to your source.war webarchive

  • '''Option 2:''' Copy the source.war file to //GLASSFISH///domains///YOURDOMAIN///autodeploy directory, glassfish will try to deploy it "automagically".
  • '''Option 3:''' Use CLI from //GLASSFISH// directory:

./bin/asadmin deploy /path/to/source.war

Deploy the modified .war file in tomcat:

  • just copy the source.war file to //TOMCAT_INSTALL///webapps directory.

Optional setup of security manager for Tomcat

On some Linux distributions you need to setup permissions for source root and data root. Please check your Tomcat documentation on this, or simply disable (your risk!) security manager for Tomcat (e.g. in debian/ubuntu : in file /etc/default/tomcat5.5 set TOMCAT5_SECURITY=no). A sample approach is to edit /etc/tomcat5.5/04webapps.policy (or /var/apache/tomcat/conf/catalina.policy) and set this (it will give OpenGrok all permissions) for your OpenGrok webapp instance:

grant codeBase "file:${catalina.home}/webapps/source/-" {     
permission java.security.AllPermission;};
grant codeBase "file:${catalina.home}/webapps/source/WEB-INF/lib/-" {     
permission java.security.AllPermission;};

Alternatively you can be more restrictive (haven't tested below with a complex setup (e.g. some versioning system which needs local access as CVS), if it will not work, please report through Discussions.

grant codeBase "file:${catalina.home}/webapps/source/-" {  
permission java.util.PropertyPermission "subversion.native.library", "read";  
permission java.lang.RuntimePermission "loadLibrary.svnjavahl-1?;  
permission java.lang.RuntimePermission "loadLibrary.libsvnjavahl-1?;  
permission java.lang.RuntimePermission "loadLibrary.svnjavahl";  
permission java.util.PropertyPermission "disableLuceneLocks", "read";  
permission java.util.PropertyPermission "catalina.home", "read";  
permission java.util.PropertyPermission "java.io.tmpdir", "read";  
permission java.util.PropertyPermission "org.apache.lucene.lockdir", "read";  
permission java.util.PropertyPermission "org.apache.lucene.writeLockTimeout", "read";  
permission java.util.PropertyPermission "org.apache.lucene.commitLockTimeout", "read";  
permission java.util.PropertyPermission "org.apache.lucene.mergeFactor", "read";  
permission java.util.PropertyPermission "org.apache.lucene.minMergeDocs", "read";  
permission java.util.PropertyPermission "org.apache.lucene.*", "read";  
permission java.io.FilePermission "/var/lib/tomcat5/temp", "read";  
permission java.io.FilePermission "/var/lib/tomcat5/temp/*", "write";  
permission java.io.FilePermission "/var/lib/tomcat5/temp/*", "delete";};
grant codeBase "file:${catalina.home}/webapps/source/WEB-INF/lib/-" {  
permission java.util.PropertyPermission "subversion.native.library", "read";  
permission java.lang.RuntimePermission "loadLibrary.svnjavahl-1?;  
permission java.util.PropertyPermission "disableLuceneLocks", "read";  
permission java.util.PropertyPermission "catalina.home", "read";  
permission java.util.PropertyPermission "java.io.tmpdir", "read";};
grant codeBase "file:${catalina.home}/webapps/source/WEB-INF/classes/-" {  
permission java.util.PropertyPermission "subversion.native.library", "read";  
permission java.lang.RuntimePermission "loadLibrary.svnjavahl-1?;  
permission java.util.PropertyPermission "disableLuceneLocks", "read";  
permission java.util.PropertyPermission "catalina.home", "read";  
permission java.util.PropertyPermission "java.io.tmpdir", "read";};

OpenGrok

Clone this wiki locally