home  /   who am i  /   my work  /   soap box

weld


download

Weld can be downloaded below. To start using weld, follow the install instructions found below.

Description Version Last change Download
Weld Build System v1.1 August 18, 2015 Tarball / Zip / GitHub
Weld Example Project v1.1 August 18, 2015 Tarball / Zip / GitHub

manual


1. Overview

Weld is a non-recursive build system on top of GNU Make. By not using make recursion, Weld can expose a friendly interface for defining build objects while maintaining perfectly parallelization, fast incremental builds, and easy extensibility. Build objects are automatically discovered: the act of simply creating a component definition file will include it in the build. Dependencies for the build object will be automatically discovered as well.

Weld strives to be platform and compiler agnostic. Build component definitions include no compiler or platform specific settings. It has been tested using GCC and MinGW on Linux, FreeBSD, and Windows. Weld is extensible which makes adding new compilers or platforms relatively painless.

An example build component definition which creates an executable named 'my_binary' can be seen below:

def.mk

name := my_binary
lang := c
type := bin

# Third party paths
include_path_list := /ext/webm/include
lib_path_list     := /ext/webm/lib

# Source files to be built
source_list := main.c gui.c protocol.c connection.c

# Files to be installed into the bin directory as well
resource_list := images/button.jpg images/cat.jpg

# Libraries built from source
source_lib_list := my_webm_helpers

# Third party libs
lib_list := webm

2. Examples

An example Weld project is visible on GitHub or available for download. Please find some examples for specific portions of the Weld build system below.

Example Description Link
Project top-level makefile weld_example/makefile
C static library definition weld_example/mylibs/fib/def.mk
C++ binary definition weld_example/mybins/runlucas/def.mk
Resource definition weld_example/myres/def.mk
Custom pseudo-targets definition weld_example/mylibs/def.mk

3. Installing Weld

The Weld build system can be installed by downloading and extracting the weld makefiles. It doesn't matter where the build system is extracted to so long as it's accessible. This path will be referred to throughout this manual as '$weld_path.'


4. Projects

A Weld project consists of any number of binaries, libraries, and resources. Weld will manage the dependencies between them and provide an elegant command line for selecting which pieces to build. Indeed, it's possible to have a single Weld project for the source code of many unrelated pieces of software.


4.1. Top-level Makefile

Each Weld project will have a single top-level makefile. This makefile invokes the weld build system. All component definition files named 'def.mk' will be discovered automatically and included in the build.

The project top-level makefile should have the following form:

makefile

# Specify any variables to be available globally
built_by   := $(shell whoami)
build_date := $(shell date +%Y%m%d)


# weld_path refers to the location Weld was installed to.  Including this file invokes the
# build system
include $(weld_path)/weld.mk

The build is always invoked by running make on this makefile. Please see the Weld command line section for details.


4.2. Component Definitions

A component definition describes a build object in the Weld project. This build object may be a binary, library, or resource file (such as an image). A component definition file is always named 'def.mk' and is placed at the top of the component's directory. A typical Weld project might be laid out as follows:

|---+ my_source
    |---- makefile
    |---+ my_libs
        |---+ foo
            |---- def.mk
            |---+ include
                |---- public_interface.h
            |---+ source
                |---- foo.c
        |---+ bar
            |---- def.mk
            |---+ include
                |---- public_interface.h
            |---+ resource
                |---- button.jpg
                |---- smiley.jpg
            |---+ source
                |---- bar.c
    |---+ my_bins
        |---+ foobar_unit_test
            |---- def.mk
            |---+ resource
                |---- strings.txt
            |---+ source
                |---- main.c

When make is invoked, these component definition files are discovered and build targets are automatically defined.


4.2.1. C/C++ Definition File

C and C++ definition files specify the dependencies of either a library or binary to be built. Weld will assume a specific file layout for these types of component definitions:

  1. The component definition file 'def.mk' lives in the top of the component directory, e.g. 'project/foo/def.mk'
  2. All source files live in the 'source' directory, e.g. 'project/foo/source/...'
  3. Any header files which are public and may be included by other components live in the 'include' directory, e.g. 'project/foo/include/...'

When a build is performed, header files which are public will be copied to a global include path within the build directory. These header files will be placed in a directory named after the component, e.g.: 'build_path/include/foo_component/public_header.h'. Other components may then include this header via '#include <foo_component/public_header.h>'

Weld looks for the following variables when including C and C++ definition files:

Variable Name Description Example
name Name the component. Many other names are derived from what is set here. For instance, a library will have the name 'lib$(name).a' and public headers will be installed to 'build_path/include/$(name)' name := foobar
type Set to either 'lib' or 'bin'. Specifies whether or not the component should be built as a library or a binary type := lib
lang Set to either 'c' or 'cpp'. Specifies whether the source should be compiled as C or C++ lang := c
def_source_subdir The directory source files live in for this component, default path is 'source/' def_source_subdir := src
def_include_subdir The directory public header files live in for this component, default path is 'include/' def_include_subdir := inc
def_resource_subdir The directory resource files live in for this component, default path is 'resource/' def_resource_subdir := res
header_list Set to the list of files to be installed as public headers. These files must live in the 'include/' directory header_list := public.h fun.h
source_list Set to the list of files to be compiled. These files must live in the 'source/' directory source_list := main.c gui.c
definition_list A list of preprocessor definitions to set for this component definition_list := COMPONENT_NAME=$(name) TURN_ON_ASSERTS
include_path_list A list of include paths. By default the include path will be set such that public headers from other components will be available. This is intended for including headers which exist outside the Weld project include_path_list := /ext/google/protobuf/include /ext/google/webm/include
source_lib_list Specify the list of libraries to link against. These should be libraries that are built by the Weld project -- those that are built from source. Weld will automatically figure out the correct build order source_lib_list := my_bar_lib my_baz_lib
lib_list Specify a list of third-party libraries to link against. These are libraries which exist outside of the Weld project -- those that are not built from source lib_list := webm protobuf
lib_path_list Specify a list of library paths where Weld can find third-party libraries lib_path_list := /ext/google/protobuf/lib /ext/google/webm/lib
compiler_flag_list Inject compiler flags into the build. Note: Weld tries to be compiler and platform agnostic. Setting this may introduce some assumptions about the environment compiler_flag_list := -Wno-crappy-warnings
link_flag_list Inject linker flags into the build. Note: Weld tries to be compiler and platform agnostic. Setting this may introduce some assumptions about the environment link_flag_list := -Wl,foobar
def_deps An arbitrary list of dependencies that should be added to all build artifacts for this component def_deps := /some/file/dep /some/other/dep

4.2.2.Resource Definition File

A resource definition file specifies a list of arbitrary files to be copied to the build's binary directory. These files might be data sets, images, configuration files, or any other files which should be included with the built Weld project.

A resource definition file may be combined with a C/C++ definition file. This means a C/C++ definition may set 'resource_list' to specify a list of resources.

Weld will assume a specific file layout for these types of component definitions:

  1. The component definition file 'def.mk' lives in the top of the component directory, e.g. 'project/foo/def.mk'
  2. All resource files live in the 'resource' directory, e.g. 'project/foo/resource/...'

Weld looks for the following variables when including resource definition files:

Variable Name Description Example
name Name the component name := my_resources
resource_list Set to the list of resources to be installed in the build's bin directory. These files must live in the 'resource/' directory resource_list := images/button.jpg data/training.dat

4.2.3. Custom Definition File

When Weld starts a build, it finds all files named 'def.mk' and blindly includes them. This means any component definition file may specify valid GNU Make instructions. This allows for some creativity and cleverness to be incorporated into a project's build. A few possibilities can be seen below.

Define pseudo targets for subsets of the build by placing a component definition file anywhere within the Weld project's directory tree:

def.mk

# Specify pseudo targets that can be invoked via 'make pseudo_target_name'
.PHONY: test_apps
test_apps : gui_unit_test server_unit_test foolib_unit_test

.PHONY: libs_only
libs_only : foolib barlib blazlib

.PHONY: monitoring_tools
monitoring tools : net_monitor status_monitor file_monitor

Specify rules to build a source file in a C component definition file:

def.mk

name := mylib
lang := c
type := lib

source_list := gen_me.c static.c

# $(def_path) is set by Weld before including this def.mk
$(def_path)/source/gen_me.c :
    echo const char* = \"Hello World\"\; > $@

Adding a custom build recipe to the global 'build' target:

def.mk

.PHONY: my_target
my_target :
    echo Buld started by $(shell whoami) >> /var/log/builds.log

# Add the target to the global build
build : my_target

4.2.4. Variables Set by Weld

Weld will set a handful of variables before including a component definition file. The definition file may use these variables as it sees fit. A list of these variables can be seen below:

Variable Name Description Example Value
def_file The complete path of the file component definition file libs/foolib/def.mk
output_path The directory the build is being output to my_build/linux/i686/debug/
bin_output_path The directory binaries are being output to my_build/linux/i686/debug/bin
lib_output_path The directory libraries are being output to my_build/linux/i686/debug/lib
header_output_path The directory headers are being output to my_build/linux/i686/debug/include
resource_output_path The directory resources are being output to my_build/linux/i686/debug/bin

4.3.Project Config Files

By default, a Weld project will include the configuration file named 'config.mk' included in the Weld installation. This sets reasonable defaults for most of the Weld configuration variables. However, if the variable 'config_file' is set prior to including the weld.mk file then that configuration will be included before applying defaults. This variable could be set in the project top-level makefile, as an environment variable, or on the make command line. A typical configuration file might look something like this:

my_config.mk

# Variables Weld looks for
source_path := . # Source for this project is in the same directory as make is invoked in
build_path  := /my_builds/my_project
arch        := amd64
mode        := debug
shell_name  := sh
platform    := unix
c_toolchain := gcc

# My variables that should be visible in every component definition file
turn_on_verbose_logging := 1
profile_points_enabled  := 1

Alternatively, a Weld project could just accept the defaults for the configuration variables and override them as-needed on the command line (e.g. make arch=i686). Below is a list of Weld configuration variables:

Variable Name Description Possible values
source_path The top-most directory of the Weld project .
build_path The directory the build should be ouput to my_build/linux/i686/debug/
arch The architecture compilations should target i686, amd64
mode The type of build to perform debug, release
shell_name The shell Weld should use sh, cmd
platform The platform to target unix, win32
c_toolchain The compilation toolchain to use gcc, clang
source_subdir The default directory to assume source files specified in source_list live source
include_subdir The default directory to assume header files specified in header_list live include
resource_subdir The default directory to assume resource files specified in resource_list live resource
show_progress Print each target being made as the build runs 1 or unset/empty

5. Command Line

Starting a build is as simple as invoking make from a project's top-level makefile. An optimal alias for building a Weld project might look like this:

Prompt> alias weld="make -C /path/to/project/ --quiet show_progress=1 -j"

The above alias will invoke make on the project's top-level directory, where the project's makefile should live. The 'quiet' option will prevent any build commands from being echoed to the terminal while enabling 'show_progress' will print each target name as it's built. The 'j' will tell make to spawn jobs in parallel.

Either via make or the above alias, the following sorts of builds may be invoked from the command line:

# Build everything
Prompt> weld

# Clean everything
Prompt> weld clean

# Rebuild the project
Prompt> weld clean build

# Build a component named 'foobin' along with it's dependencies
Prompt> weld foobin

# Clean just a component
Prompt> weld clean_foobin

# Build just the component's resources
Prompt> weld foobin_resources

# Clean just the component's resources
Prompt> weld clean_foobin_resources

# Build just a single source file of the component
Prompt> weld foobin/main.o

# Build just the headers for a single component
Prompt> weld foolib_headers

# Build just a library and it's dependencies
Prompt> weld libfoolib.a

# Build a pseudo target defined in a custom component definition
Prompt> weld my_psuedo_target

# Build overriding the target mode
Prompt> weld mode=debug

# Build overriding the mode and architecture
Prompt> weld mode=release arch=i686

# Build specifying a custom config_file
Prompt> weld config_file=my_config.mk

# Build overriding the build output path
Prompt> weld build_path=/my/one-off/builds

6. Licensing

Weld is free software. Andrew Gottemoller licenses the software on this page to you under the terms of the GNU General Public License as published by the Free Software Foundation; you can redistribute it and/or modify it under the terms of the GNU General Public License either version 3 of the license, or (at your option) any later version.

There is NO WARRANTY for Weld, express or implied, including the implied warranties of MERCHANTIABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU Public License along with Weld. If not, see <http://gnu.org/licenses/>.


7. Alternative Licensing

A special license for Weld is available if you are unable to meet the conditions of the license described above. For more information, please contact me.