Per project management and workflow

Motivation

OpenGrok can be run with or without projects. A project is simply a directory directly underneath the OpenGrok source root directory. A project can have zero or more Source Code Management repositories underneath. In a setup without projects, all of the data have to be indexed at once. With projects however, each project has its own index so it is possible to index projects in parallel, thus speeding the overall process. Better yet, this can include project synchronization which usually involves running commands like git pull in all the repositories for given project.

This is handy in case the synchronization, indexing for some of the projects is taking a long time or simply you have lots of projects. Or all of it together.

Previously, it was necessary to index all of source root in order to discover new projects. Starting with OpenGrok 1.1, using the opengrok-projadm tool (that utilizes the opengrok-config-merge tool and RESTful API) it is possible to manage the projects. As a result, the indexing of complete source root is only necessary when upgrading across OpenGrok version with incompatible Lucene indexes.

The following is assuming that the commands opengrok-projadm, opengrok-groups and opengrok-config-merge tools are in PATH. You can install these from the opengrok-tools python package available in the release tarball.

Combine these procedures with the parallel processing tools (see repository synchronization) and you have per-project management with parallel processing.

The following examples assume that OpenGrok install base is under the /opengrok directory.

Workflow

TODO

Building blocks

Configuration backup

The next sections start by suggesting to backup current configuration. This could be done by e.g. copying the configuration.xml (that is written by the indexer when using the -W indexer option) file aside, taking file-system snapshot of the directory the configuration is stored in etc.

This is necessary as a prevention if something goes wrong.

Indexing a project

TODO

Adding and indexing a project

• backup current config
• add the project data to a directory under the source root directory
  • this usually involves running VCS command such as git clone, extracting source code from an archive, etc.
• perform any necessary authorization adjustments
• add the project to configuration (also refreshes the configuration on disk):

   opengrok-projadm -b /opengrok -a PROJECT

• change any per-project settings (see Web services)
• index the project
  • it is recommended to use the opengrok-reindex-project script (it downloads fresh configuration from the webapp)
• save the configuration (this is necessary so that the indexed flag of the project is persistent). The -R indexer option can be used to supply path to read-only configuration so that it is merged with current configuration.

   opengrok-projadm -b /opengrok -r

• now it is possible to reindex the sync and reindex the project in a cycle (see repository synchronization)

Deleting a project

• backup current config
• delete the project from configuration (deletes project's index data and refreshes on disk configuration). The -R indexer option can be used to supply path to read-only configuration so that it is merged with current configuration.

   opengrok-projadm -b /opengrok -d PROJECT

• perform any necessary authorization, group, per-project adjustments in read-only configuration (if any)
  • see putting read-only configuration into effect