Sandbox Documentation

Sandbox Concepts

A sandbox represents a developer's private build area. To avoid the need to compile an entire project, only the small pieces currently being modified by the developer are normally copied to the user's sandbox. The remainder of the build environment comes from one (or more) backing trees to which the user has read-only access. The backing tree will hold an image of the source and compiled objects and libraries for the target architecture.

The use of backing trees requires some support from the underlying build system and appropriate construction of Makefiles (or their equivalent). Systems with the needed functionality include GNU Make, CMake and Scons. The fundamental ideas are to separate the output directories for objects from the directories holding the source code and exploit a mechanism like GNU Make's VPATH to search multiple directories for a required dependency.

Note: while it is typical for a sandbox to be backed by a single backing tree, this implementation permits multiple backing trees to be chained. This is often used when developers are collaborating on experimental or incompatible changes; an intermediate backing tree is used to share work in progress and override images that would normally have come from a production backing tree.

At a given moment in time, a sandbox is typically associated with a single target architecture. It can be rebased to a different target architecture using the resb command and the sb_retarget command will automatically be run on the next workon invocation to regenerate any architecture-specific configuration files.

A developer may use multiple sandboxes to keep progress on independent work units separated; common best practice is to pursue work on only one feature or defect in a given sandbox, though sometimes work units are interrelated. It is almost always a bad idea to tackle an emergency bug fix in a sandbox were work was already underway; these unscheduled but focused work activities should be handled in a distinct sandbox that is removed after the bug fixes are committed. Sandboxes can be rebased against historical backing trees to compile a new version of previously deployed executables and libraries while remaining compatible with the data structures as they were defined for those releases. This is most useful when releasing object-code-only distributions and one needs to apply bug fixes against releases previously shipped to customers, which may have incompatible data structure layouts and function names, etc. that differ from the current source code images.

Sandbox Commands

A user's sandbox descriptions are stored in a sandbox rc file, typically $HOME/.sandboxrc. It can be overridden by the -rc fileName argument on most commands. While logically a “database” that maintains a list of sandboxes and their associated attributes, this is realized via a simple text file structure and it can be modified by a text editor if the need arises.

The following commands are used when working with sandboxes. Each command is described in detail later.

mkbt
make a backing tree (prepares directory structure).
mksb
make a new sandbox.
resb
rebase an existing sandbox.
rmsb
remove a sandbox (removal of the directory tree is left to the user).
lsbt
list available backing tree names or paths.
lssb
list available sandboxes.
workon
create subshell and start working in a sandbox.
buildArch
list default or available build architectures; used to set compiler target architecture.
getsbattr
get sandbox attributes from sandbox rc file.
setsbattr
set sandbox attributes in sandbox rc file.
delsbattr
delete sandbox attributes from sandbox rc file.
sbinfo
display information about the active workon session.
pathedit
prefix/append/remove elements from a character-separated list .
findFile
find a file within a collection of directories.
findFileInSB
find a file within a collection of sandbox trees.
insb
in a sandbox, run a command using automatically located file name arguments.
sbdiff
compare local version of a file against previous.
sbmake
sandbox-aware cover to make utility.
sbrel
display relative components of sandbox-related path.
sbscan
scan files in sandbox and backing trees for text pattern.
sbsvnget
sandbox-aware interface to Subversion that supports sparse checkouts.
sb_retarget
regenerate any target-specific configuration files after a sandbox has been rebased.


Common arguments:

-sb
sandbox name
-bt
name of a backing tree, which can be fully qualified or assumed to name one under the backing tree root.
-ba
target build architecture; defaults to that indicated by buildArch, but can be used to select different sets of compiler options, such as optimized vs. debug with profiling or cross-compilation.
-btroot
the root of all backing trees. Can be set by the BT_ROOT_DIR environment variable and defaults to /opt/backingTrees.
-sbroot
the root of the user's sandbox collection. Can be set by the SB_ROOT_DIR environment variable and defaults to $HOME/sandboxes.
-rc
specifies the sandbox rc file. Defaults to $HOME/.sandboxrc.
-s
Usually used to specify a separator character.

mkbt – make backing tree

A backing tree directory structure can be prepared using the mkbt command. Depending on the administrative environment, this command might normally only be used by an administrator preparing shared resources, but it can be used by a single developer who specifies a backing tree root under their respective home directory. Most sandbox users will not have occasion to use the mkbt command. The most common usage of this utility is from automatically run scripts created by an administrator that setup a build tree, extract current source from a source code repository and perform a compile.

usage: mkbt [ btroot btRoot] [ ta targetArchitecture] ...  bt backingTree [...]

-btroot = specify alternate backing tree root, default is "/opt/backingTrees"

-ta = target architecture, default is native;

"all" means prepare for all supported

"avail" means prepare for all locally supported

The root of backing tree is normally obtained from BT_ROOT_DIR, which defaults to /opt/backingTrees if not set.

The mkbt command prepares several architecture-specific directories and configuration files. If no target architecture is specified, only configuration related to the local native build architecture is performed. The alias of “avail” can be used to set up the configuration for all locally-supported architectures. The alias of “all” can be used to set up the configuration for all known architectures.

In some environments, there may be no distinction between these three options. However, a not atypical case would be an environment where a cross-compiler exists for Windows on Linux x86 systems and is not present on Sun SPARC hardware running Solaris nor PowerPC hardware running AIX or Linux. Similarly, native compilation by Microsoft Visual Studio might be a known platform, but it would not be performed by the Linux or Solaris hosts. If the backing tree is to be visible to all platforms (requiring it to be exported as a networked file system), then using “-ta all” is arguably the best choice. If hosts have their own copies of backing trees, then “-ta avail” could be preferred.

mksb – make a sandbox

A user can create a new sandbox tree using the mksb command. A sandbox root starting with the user's HOME directory will be altered to replace that prefix with “~”. This permits the sandbox configuration file to be used on alternate hosts that might mount the network file system holding the user's home directory in a different location. This issue frequently arises when one host “owns” the user's home directory and other hosts (perhaps running different operating systems) have different naming conventions.

usage: mksb [ i] [ rc rcFile] [ btroot btRoot=/opt/backingTrees] [ sbroot sbRoot=/home/geoff/sandboxes] [ desc description] [ ta targetArch] [ force]  sb sandboxName  bt backingTree [...]

-i = interactive mode

-rc = specify alternate rc file, default is "$HOME/.sandboxrc"

-btroot = specify alternate backing tree root, default is "/opt/backingTrees"

-sbroot = specify alternate sandbox root, default is "$HOME/sandboxes"

-desc = descriptive note for sandbox to help identify its purpose

-ta = desired target architecture

-force = allow mksb to tolerate existing sandbox directory

resb – rebase a sandbox

The resb command can be used to rebase an existing sandbox. Normally installed as an alias for mksb. If already in a workon shell, the  sb sandbox argument may be omitted and will be inferred to be the current sandbox.

usage: resb [-i] [-rc rcFile] [-btroot btRoot=/opt/backingTrees] [ sbroot sbRoot=/home/geoff/sandboxes] [ desc description] [ ta targetArch]  sb sandboxName  bt backingTree [...]

-i = interactive mode

-rc = specify alternate rc file, default is "$HOME/.sandboxrc"

-btroot = specify alternate backing tree root, default is "/opt/backingTrees"

-sbroot = specify alternate sandbox root, default is "$HOME/sandboxes"

-desc = descriptive note for sandbox

-ta = desired target architecture

sb_retarget – do reconfiguration after an sandbox has been retargeted

Some build environments may require configuration files to be updated when a sandbox's target architecture has been changed. The sb_retarget utility is an internal script that will automatically be run by the first workon performed after a resb command alters the target architecture (using the -ta argument). The current implementation deals with regeneration of Makefiles derived from CMakeLists.txt files; if the build system does not use cmake, the script has no effect.

rmsb – remove a sandbox

The rmsb command is used to remove an existing sandbox. It is normally installed as an alias for mksb. An existing sandbox's corresponding entries will be deleted from the sandbox rc file; however the local sandbox directory structure will not be removed unless the -force flag is specified. If more than one sandbox name referenced the same sandbox tree, rmsb will refuse to delete the sandbox tree.

usage: rmsb [ i] [ rc rcFile] [ force]  sb sandboxName

-i = interactive mode

-rc = specify alternate rc file, default is /home/user/.sandboxrc

lsbt – list available backing trees

A list of available backing trees can be obtained with the lsbt command. The backing tree names are normally separated by colons, but this character can be overridden with the -s option.

usage: lsbt [{ n| f}] [ s separator] [ btroot backingTreeRoot=/opt/backingTrees]

-n = list only backing tree name

-f = list full path name of backing tree (default)

-btroot = backing tree root, default is "/opt/backingTrees"

default output separator: ":"

lssb – list user's sandboxes

The sandboxes maintained by a user can be obtained with the lssb command. One sandbox name is displayed per output line. If verbose mode is selected with the -v option, the root directory and description of the sandbox will also be displayed.

usage: lssb [ v] [ rc rcFile]

-rc = alternate rc file, default is $HOME/.sandboxrc

sbinfo – display information about current sandbox

The sbinfo utility is a convenience program used to display information about the current sandbox.

The output looks similar to:

sandbox: sbconv targetArch: Linux.x86_64 buildArch: Linux.x86_64

workon – Enter a sandbox work environment

The workon command is the most frequently used sandbox-related command because one only creates a sandbox once, but will frequently workon it. The workon command can be used to run commands in the context of a sandbox, but usually the command argument will not be specified. If a command is not specified, a sub-shell is created with the appropriate environment. The shell to be used will be taken from the SB_SHELL environment variable; if this is not set, the current SHELL will be used.

The SB_USE_SSH_AGENT environment variable can be set to either 1 or 0 (it defaults to 0). If set to 1 or -ssh is specified, then an ssh-agent is automatically started if one was not already available and the user's ssh keys are added to it. In contrast, the -nossh option prevents the launch of an ssh-agent and loading of the user's ssh keys. In development environments that make use of cross-machine file transfers and have the appropriate public keys distributed amongst the host population, SB_USE_SSH_AGENT will normally be set to 1.



usage: workon  sb sandboxName [ ta targetArchitecture] [ bt backingTree] [ ba buildArchitecture] [ rc rcFile] [{-ssh | -nossh}] [cmd ....]

-ta = specify target architecture

-rc = override sandbox rc file, default="$HOME/.sandboxrc"

-bt = override backing tree

-ba = specify build architecture (not normally used)

-ssh = setup ssh keys and start ssh-agent if needed

buildArch – report information about supported architectures

The buildArch command is used to display the available target architectures supported by the build infrastructure.

usage: buildArch [ s separator] [ btroot backingTreeRoot] [ ba |  ta |  all]

-all = list all available architectures

-ba = list current build architecture

-ta = list current target architecture

getsbattr – get sandbox attributes from sandbox rc file

The getsbattr command implements a standard interface for retrieving sandbox-related attributes. This interface should be used in preference to directly accessing the sandbox rc file as it permits all of the implementation details to be altered without requiring modifications to applications. The sandbox name is optional if used within a workon session as it defaults to the name of the local sandbox.

usage: getsbattr [ v] [ null] [ sb sandboxName] [ rc rcFile] attrName ...

-v = prefix results with attribute name

-null = report null string rather than [undefined]

setsbattr – set sandbox attributes in sandbox rc file

The setsbattr command implements a standard interface for modifying sandbox-related attributes. This interface should be used in preference to directly accessing the sandbox rc file as it permits all of the implementation details to be altered without requiring modifications to applications. The sandbox name is optional if used within a workon session as it defaults to the name of the local sandbox.

Attribute value pairs can be specified using contiguous elements with an equals-sign separator or as space-separated pairs.

usage: setsbattr [ sb sandboxName] [ rc rcFile] attrName=value ...

delsbattr – delete sandbox attributes from sandbox rc file

The setsbattr command implements a standard interface for modifying sandbox-related attributes. This interface should be used in preference to directly accessing the sandbox rc file as it permits all of the implementation details to be altered without requiring modifications to applications. The sandbox name is optional if used within a workon session as it defaults to the name of the local sandbox.

usage: delsbattr [ sb sandboxName] [ rc rcFile] attrName ...

pathedit – Modify a path or list

The pathedit command is a generic convenience utility that can append, prepend or remove elements from a list, typically a colon-separate list of directories.

usage: pathedit [{ a| r| p}] [ s separator] initialPath arg [...]

-a = append, -p = prepend, -r = remove

default mode: "a" separator: ":"

findFile – search a set of directory paths for a file

The findFile command is a generic convenience utility that will search a set of colon-separated directory paths for a file. If no file is found, an exit status of 1 is returned; otherwise 0 is returned. The full path name of the found files is displayed, one per line.

usage: findFile [ 1 |  all] [ s separator] fileOrDirName path [...]

path can contiguous elements, default separator is a colon

-s separator = define alternate path separator character

-1 = report first entry found only (-2 = first two, etc.)

-all = report all entries found (default)

findFileInSB – search a set of sandbox trees for a file

In the context of a sandbox, findFileInSB will search a set of sandbox trees for a file appearing at the corresponding location in the chain of sandbox trees. Any file name paths that are passed as an absolute path name (i.e., beginning with a “/”) or explicitly relative to the current or parent directory are returned as-is; no search will be performed. This utility is commonly used by scripts to locate the Makefile for a sandbox or appropriate configuration files. The -all option searches all trees whereas -allbt selects all backing trees but not the local sandbox; -sb selects the local sandbox tree, -bt1 selects for first backing tree, -bt2 selects the second backing tree, etc.

usage: findFileInSB [ q] [ s separator] [ d additionalDir] { all |  allbt |  sb |  bt1 |  bt2 | ... } ... fileName [...]

insb – invoke command using file found in sandbox hierarchy

The insb command is a convenience utility that allows one to invoke a command with file name arguments that are obtained after having located the indicated file using findFileInSB. For example, “insb vi Makefile” will start the vi text editor using the file name of the Makefile corresponding to the current location in the sandbox tree. Arguments and anything that can not be resolved to a file name will be passed as-is. There are two special cases. If a file name begins with the “@” character, the local sandbox will not be searched. If a file name begins with the “%” character, the file name portion will be used as-is. Example usage:

insb vi source.c

insb gvimdiff source.c @source.c

insb lpr -P :printer2 source.h

usage: insb command fileName [...]

NOTES: arguments beginning with "-" are passed as-is.

If a fileName begins with "%", the fileName portion is used as-is.

If a fileName begins with "@", the local sandbox is not searched.

This is useful to compare local vs. backing tree files:

insb diff source.c @source.c

sbdiff – perform diff against local copy of a sandbox file

The sbdiff command is a sandbox-aware convenience utility that is typically used to display differences between the image of a file in a local sandbox and its original state or current state in a source code repository.

usage: sbdiff [{ r revision |  local |  committed }] [ diff] [ nows] [ using prog] fileName [...]

By default, the differences between the original state of the file and its current local (-local) state are displayed to highlight the modifications that have been made. An explicit revision can be specified or the most recent committed version can be selected using the -committed flag.

If differences in white space are not intended to be significant, the -nows option can be specified.

By default, the gvimdiff program will be used to visually display the contrast between the selected files; however, the -diff option can be used to select the text-only output from the diff utility and the -using option can be used to specify an arbitrary comparison program.

sbmake – sandbox-aware interface to make program

The sbmake utility is a cover script for a make-like program that searches the sandbox and backing trees for a corresponding makefile. By default, it will search for sb.Makefile, Makefile and makefile. Note that this order is not identical to that of GNU make, which looks for makefile before Makefile. An arbitrary set of potential file names can be specified using the -f option in appropriate combinations.

The desired make command can be provided in the SB_MAKE_COMMAND environment variable or via the -m command line option; it defaults to /usr/bin/make.

Note: the absolute path is used to avoid any possibility of sbmake recursively invoking itself, but this does require that the SB_MAKE_COMMAND environment variable be set appropriately if the make program to be used is not /usr/bin/make.

usage: sbmake [ m makeCmd] [ C dir] [ f file][...]

Developers who do not want to add sbmake into their repertoire of commands can add an alias for make to sbmake, but this is perhaps best done in $STS/BuildEnv/shell_rc.csh:

setenv SB_MAKE_COMMAND `which make`

alias make sbmake

or $STS/BuildEnv/shell_rc.bash:

SB_MAKE_COMMAND=`which make; export SB_MAKE_COMMAND

alias make sbmake



The sbmake utility works outside of the context of a sandbox, so it is feasible to use it in situations where make would normally have been invoked. For example, as the makeprg in $HOME/.vimrc:

:set makeprg=/opt/backingTrees/bin/sbmake\ $*



sbrel – report common relative path

The sbrel utilty outputs the subtree path below a sandbox or backing tree. It is most often used on the command line to conveniently switch between directories, but can be used by scripts to determine the context of a directory path. The path to be evaluated is normally taken from the current working directory, but can be provided as a command line argument. The -showroot argument can be used to output the root of the path rather than the common subtree component. In general, one should expect `pwd` to be equal to `sbrel -showroot``sbrel`.

usage: sbrel [ showroot] [completeDirPath]

Examples:

cd $BT1`sbrel` # change to directory under $BT1

cd $STO`sbrel` # change to object directory

cd $STS`sbrel` # change to sandbox directory

sbscan – search sandbox and backing trees for text pattern in files

The sbscan utility is effectively a sandbox-aware combination of the find and egrep utilities. The default -top option will search from the root of each selected sandbox and backing tree, whereas the -rel option restricts the search to only the same relative subtree across the selected sandbox and backing trees. If the default -partial option is used, only the portion of the path below the root of search trees is output (note that the root will vary depending on whether -top or -rel was in effect). In contrast, the -full option outputs the entire path name of a file. The -suffix option can be used to specify a file name pattern; it defaults to '*.hc*', which is intended to select C/C++ source and header files.

A set of sandbox and backing tree roots can be specified using -all or combinations of -sb, -allbt, -bt1, -bt2, etc. If none are specified, the default behavior is to search only the local sandbox.



usage: sbscan [ top |  rel] [ full |  partial] { [ all] | [ allbt] |[ sb] [ bt1] [ bt2] [ bt3] [ bt4] } [ d dir] ... [ o file] [ suffix fileNamePattern] pattern ...

-top = start search from top of tree

-rel = start search from relative position within tree

-full = emit full path names

-partial = emit partial path below tree root

-all = search sandbox and all backing trees

-allbt = search all backing trees

-sb = search sandbox

-btN = search backing tree N in chain

-d dir = add an arbitrary tree to be searched

-o file = also output result into file

-suffix = file name search pattern

Default file name pattern is *.[hc]*

Default is start search from top of tree

Default is to emit partial path below tree

Default is to just search local sandbox

Examples:

# display all makefiles that reference CXXFLAGS

sbscan -suffix '*akefile' -all CXXFLAGS

# display all makefiles under the current working directory

# that reference CXXFLAGS

sbscan -suffix '*akefile' -all -rel CXXFLAGS

sbsvnget – Get files from Subversion

The sbsvnget utility is a sandbox-aware interface to Subversion that can make appropriate svn checkout and update invocations to retrieve selected source files. Normally, a Subversion checkout will retrieve an entire source tree, which defeats one of the benefits of using sandboxes. If normal svn checkout requests are done on subsets of the source tree, they appear as distinct, unrelated checkouts and commands like svn status, svn diff, svn commit, etc. would have to be invoked against the distinct checkout roots.

In contrast, sbsvnget enables retrieval of just the files of interest, while keeping the directory structure under the sandbox as a single Subversion checkout. The sbsvnget utility is aware of relative position within a sandbox tree and ensures that the directory hierarchy in the sandbox is in parallel with that of the repository.

The host and root path of the repository can be specified via environment variables (SVN_HOST and SVN_PATH), sandbox-specific attributes (sb_svnServer and sb_svnRoot, set by the user via setsbattr) or command-line arguments (-server and -path).

usage: sbsvnget [ server modeAndhostName] [ path pathToRepository] [-force] dirOrFileName ...

The -force option can be used to take over an existing directory tree. It is not normally used, but is useful when a sandbox has already had directories and some new files added prior to the first sbsvnget.

Sandbox Environment Variables

Most of the environment variables documented below are set by the workon command, but there are a handful reserved for use by an administrator or user.

Administratively Set Variables

The two key exceptions are the BT_ROOT_DIR and SB_ROOT_DIR variables, which are never set by any of the commands. They are reserved for use by an administrator or user to point the framework at alternate locations. The SB_MAKE_COMMAND variable is also not set by any script; if /usr/bin/make is not in use, this variable must be set appropriately for correct operation by sbmake.

BT_ROOT_DIR
Specifies root of all master backing tree collection; defaults to /opt/backingTrees. A user or administrator can set this environment variable to override the default backing tree root directory on the local system. All commands that reference this directory take a -btroot argument which can also be used to achieve the required effect.
SB_ROOT_DIR
Specifies root of user's sandbox collection; defaults to $HOME/sandboxes. A user can set this environment variable to override the user's default sandbox collection root. All commands that reference this directory take a -sbroot argument which can also be used to achieve the required effect, but this is error-prone and intended for use by scripts rather than humans.
SB_MAKE_COMMAND
Specifies the full path name of the make (or equivalent) program to be used; if not specified, it defaults to /usr/bin/make. If the build system in use does not use /usr/bin/make, SB_MAKE_COMMAND should be set appropriately to obtain correct operation from sbmake.
SB_SHELL
Specifies the full path of the command shell to use in a workon session; if not set, defaults to $SHELL.
SB_USE_SSH_AGENT
The SB_USE_SSH_AGENT environment variable can be set to either 1 or 0 (it defaults to 0). If set to 1, then an ssh-agent will be started if necessary and the user's ssh keys will be loaded. In development environments that make use of cross-machine file transfers and have the appropriate public keys distributed amongst the host population, SB_USE_SSH_AGENT will normally be set to 1.

Automatically set variables

The following variables are automatically set by workon.

SB_NAME
Indicates the short name of current sandbox.
SB_BUILD_ARCH
Identifies the local compilation architecture of sandbox environment.
SB_TARGET_ARCH
Identifies the target architecture for sandbox environment. Usually, this is a name like Linux.x86_64 or SunOS.sparc, but virtual architectures that represent a collection of compilation settings can be created by appending administrative-defined text. Example: Linux.x86_64.testCoverage, Linux.x86_64.Debug.
BT_PATH
Specifies the absolute path names of all backing trees associated with the sandbox. Normally, this is a colon-separated list; however, some platforms may use an alternative separator (such as a semicolon).
BT1
Convenience variable that specifies the root directory of first backing tree associated with the sandbox. It is important to remember that sandboxes are really backed by a path of backing trees (BT_PATH) and are not restricted to a single root directory. Scripts should almost always use BT_PATH rather than BT1 to preserve generality. If present, the second and subsequent backing tree roots are named BT2, BT3, etc.
ST
Specifies the root directory of local sandbox, which should be $SB_ROOT_DIR/$SB_NAME.
STS
Specifies the root directory of local sandbox source tree, which should be $ST/src.
STO
Specifies the root directory of local sandbox object tree for the current build target architecture, which should be $ST/obj/$SB_TARGET_ARCH.
STI
Specifies the root directory of local sandbox exported include files, which should be $ST/include.
STL
Specifies the root directory of local sandbox exported libraries for the current build target architecture, which should be $ST/${SB_TARGET_ARCH}/lib.
STB
Specifies the root directory of local sandbox exported binaries for the current build target architecture, which should be $ST/${SB_TARGET_ARCH}/bin.

NOTE: the SB_BUILD_ARCH and SB_TARGET_ARCH environment variables are often identical; however, they will always be different when utilizing support for a cross-compiler (e.g., compiling for Windows targets on an x64 Linux host). The feature can also be exploited to support incompatible compilation options, such as enabling various profiling options or compiling for a different addressing mode (e.g., 32-bit vs. 64-bit). In general, SB_BUILD_ARCH will be used to find utilities used during the build process by the local machine, like deployment utilities. It may also include specification of utilities such as the compiler, assembler, linker, etc., but these are often associated with the target architecture.

Configuration Files

Various global configuration files are maintained in the $BT_ROOT_DIR/config directory. These specify the supported architectures, customization scripts for the build and target environment, and shell-specific rc files. When a new backing tree is created, symbolic links of the appropriate files are made from the $BT_ROOT_DIR into the backing tree's $BT/BuildEnv directory. This permits each backing tree to have custom configuration files, but they start out using common images. Consequently, normally any updates to the master versions will have immediate effect in all backing trees except those in which effort was made to create unique copies.

Note: backing trees are identified by the existence of their BuildEnv directory; the lsbt command uses the presence of a BuildEnv directory to find appropriate trees rooted under $BT_ROOT_DIR.

Supported Architectures

The list of all available target architectures is provided in the ALL_supportedArchitectures.txt file. The subset of locally supported architectures is provided in either `hostname  s`_supportedArchitectures.txt or `buildArch -ba`_supportedArchitectures.txt. In some environments, the backing trees will be placed on a network file system and visible to machines with heterogeneous operating systems and processor architectures and not all platforms will be capable of compiling for non-native targets. The three distinct files thus indicate the extent of all platforms supported by the local development infrastructure, those available on a specific machine and those supported by any of the build machines with a given operating system/processor combination.

Custom Architecture Definitions

Sometimes it is useful to define a virtual custom architecture that is associated with the definition of compiler flags which are incompatible with normal defaults (e.g., --test-coverage). This can be done by the owner of the build system by editing the config/ALL_supportedArchitectures.txt file, but a developer can define their own variants by creating an $ST/BuildEnv directory and creating one of the three Architectures.txt files. For example, one might add a Linux.x86_64.coverage entry to correspond to a build with test coverage enabled and create a $ST/config/targetEnv.Linux.x86_64.coverage_sh.txt configuration file with all of the desired configuration definitions.

Build and Target Architecture Configuration

There are several optional configuration files consumed by the workon command that can be used to customize the environment. These files all end in “_sh.txt” to indicate they are shell scripts intended to be included rather than executed directly.

The preTreeEnv_sh.txt file is used by the workon command to setup any specialization specific to the backing tree chain. Often this file will not be present and will only be used when project-specific customization is required. The corresponding postTreeEnv_sh.txt file will be processed after all other configuration files have been consumed.

The buildEnv_`buildArch  ba`_sh.txt file is used by the workon command to setup any specialization needed in the configuration environment for the build architecture. Content in this file would typically define settings related to the local build infrastructure. The similar targetEnv_`buildArch   ta`_sh.txt file is used by the workon command to setup any specialization required by the configuration for the target architecture, such as compiler name, optimization flags, machine architecture, etc. Typically, this will involve specifying an alternate C/C++ compiler in the CC and CXX environment variables and appropriate values for the TARGET_ARCH environment variable (e.g., “-m64 -fPIC”). For cross-compilation, it will also usually involve specifying an alternate linker and library archive utility.

As a final step, a sbEnv_sh.txt will be processed if present. This file is often used for developer-specific customizations.

These files are consumed by the workon command and must be valid for processing by the /bin/sh interpreter. The defined order:

  1. preTreeEnv_sh.txt

  2. buildEnv_`buildArch -ba`_sh.txt

  3. targetEnv_`buildArch -ta`_sh.txt

  4. postTreeEnv_sh.txt

  5. sbEnv_sh.txt



Shell-specific RC Files

Users can use the command shell of their choice. After preparing the sandbox environment, the workon command passes a shell_rc.`basename ${SB_SHELL}` script to the spawned subshell. The scripts must be written in the respective language for the selected SHELL. Consequently, it is suggested that the complexity of these scripts should be minimized to avoid duplicated effort and generally used only to implement shell-specific functionality, like setting a command prompt. Customizing the appropriate buildEnv_*_sh.txt or targetEnv_*_sh.txt file is the preferred mechanism as only one set of modifications will be required.

Sandbox RC File Format

The sandbox rc format appears as a collection of lines formatted as demonstrated below:

SB_${sbName}_${attribute}_ value

The ${sbName} portion corresponds to the sandbox's name and the ${attribute} portion is the name of the desired parameter. Note that there is always an underscore after the attribute name. The sandbox rc file is modified by the mksb command (and its variants resb and rmsb). It is a simple text file and can be easily edited by humans, though this will normally not be done. Instead, the mksb utility will be used to create the appropriate records. Other scripts use the getsbattr, setsbattr and delsbattr utilities to modify the sandbox rc file; this level of abstraction permits replacing the mechanism for storing sandbox attributes persistently without requiring modification of the sandbox-related commands.

The file format for the sandbox attributes supports an arbitrary number of parameters and can be extended at any time without affecting previously implemented applications. There are 3 mandatory attributes, sbname, sbroot and btpath:

sbname
which identifies the sandbox name,
sbroot
which identifies the root directory of the sandbox and
btpath
which identifies the path of backing trees associated with the sandbox. While it is common for only a single element to be present in the backing tree path, it should be stressed that a list of several backing trees is permitted. This advanced usage can be used to chain together another user's sandbox, a global backing tree with a subset of content and a master backing tree.
description
maintains a description of the sandbox to help the user recall the purpose of the sandbox.
defaultTargetArch
defines the target architecture to be used by workon if none is specified as an argument.
SandboxRetargetted
a special flag set by the resb command to indicate that a sandbox has been retargetted to a different architecture. If the workon command sees this flag, it will run the sb_retarget command and delete the attribute.

The format of a sandbox rc file is illustrated below:

SB_test1_sbname_ test1

SB_test1_sbroot_ ~/sandboxes/test1

SB_test1_btpath_ /opt/backingTrees/tree1:/opt/backingTrees/tree2

SB_test1_defaultTargetArch_ Linux.x86_64

SB_test1_description_ Sample sandbox

Sample Usage

mkbt -bt tree1

mksb -sb test1 -bt tree1 -desc “Sample sandbox”

workon -sb test1

resb -sb test1 -ta Windows.i686



Sandbox First Steps

Optional Environment Variables

To get started, one needs to know where the backing trees will be maintained. This will normally be set by the system administrator and, by default, this will be under /opt/backingTrees. If this is not the case, the BT_ROOT_DIR environment variable should be set to point at the appropriate directory. It is possible to pass a -btroot option to commands that need to be aware of the BT_ROOT_DIR, but this is extra effort and prone to error.

The SB_ROOT_DIR environment variable serves to specify the default directory under which a user's sandboxes should be created. If not set and the -sbroot command line option is not specified, the default of of $HOME/sandboxes will be used by the mksb command. Since every sandbox has its root maintained in the sandbox rc file, a sandbox can be stored anywhere.

The location of the user's sandbox rc file that stores the parameters associated with each sandbox is usually $HOME/.sandboxrc. It is possible to specify an alternative configuration file using the -rc command line option, but it would be rare to find a compelling reason to use an alternate name or file.

A user can specify the shell they wish to use in workon sessions by setting the SB_NAME environment variable to the full path of the shell (e.g., /bin/csh). If not set, the shell in use at the time of the workon invocation will be used. A typical scenario is one in which the system administrator has mandated the use of /bin/sh for login shells but a developer prefers to use /bin/csh for development work.

The sbsvnget utility can use the environment variable SVN_HOST and SVN_PATH to access a Subversion repository. If only a single repository is in use locally, it can be extremely convenient to set these variables.

Automated backing tree builds

Typically, an administrator sets up an automated process that uses the mkbt command to create a new backing tree, extract the most current source code image from a repository and do a complete build. If successful, a symbolic link with an appropriate name (e.g., current) will be created to point at the new backing tree. Developers can use that symbolic name as a backing tree and have their sandboxes automatically backed by the most recent successful build. In a similar fashion, an image of production releases is maintained, making it easy for developers to immediately compile against any production release in use by customers.

Creating a sandbox

Sandboxes are created using the mksb command. One needs to know the name of the sandbox to be created and the backing tree(s) with which it should be associated. One can get a list of the available backing trees by issuing the lsbt command. The name of the sandbox is entirely up to the developer, but must be something representable as a directory name. It should not use white space even if the local filesystem is capable of storing such names.

The mksb command has a simple interactive mode (requested via the -i flag) that will prompt for required parameters:

$ mksb -i

/opt/backingTrees/bin/mksb: sandbox name was not specified.

Enter name of sandbox: sbtest

/opt/backingTrees/bin/mksb: no backing trees specified.

Available backing trees: current test1 release

Enter backing tree name or path: current

Developers usually quickly become proficient with the mksb command and will just enter the needed arguments on the command line. A description can be added to annotate the intended usage of the sandbox if the name itself was not sufficiently informative.

mksb -sb sbtest -bt current -desc 'Feature 53126'

Once a sandbox is created, a developer makes use of it via a workon command.

Working in a Sandbox

A developer usually issues a workon command to begin a development session:

workon -sb sbtest

When a development session is complete, the shell is closed (usually via control-D or an explicit exit).

It is possible to use a workon command to invoke a command in the context of a sandbox. For example:

workon -sb sbScan -suffix makefile CXXFLAGS

This usage is most commonly found in scripts.

Each sandbox has a default target architecture, which can be altered with the resb command. It is also possible to set a different target architecture for the duration of the current workon session using the -ta option. This can be useful when a developer wants to use the same sandbox and compile for multiple machine architectures.