A Policyfile is a way to create immutable collections of cookbooks, cookbook dependencies, and attributes defined in a single document that is uploaded to the Chef Infra Server. The Policyfile is then associated with a group of nodes. When these nodes perform a Chef Infra Client run, they utilize recipes specified in the Policyfile.
Policyfiles make it easier to test and promote code safely with a simpler interface. Using a Policyfile improves the user experience and resolves real-world problems that some workflows built around Chef Infra must deal with. The following sections discuss in more detail some of the good reasons to use Policyfile, including:
The knife command line tool maps very closely to the Chef Infra Server API and the objects defined by it: roles, environments, run-lists, cookbooks, data bags, nodes, and so on. Chef Infra Client assembles these pieces at run-time and configures a host to do useful work.
Policyfile focuses that workflow onto the entire system, rather than the individual components. For example, Policyfile describes whole systems, whereas each individual revision of the Policyfile.lock.json file uploaded to the Chef Infra Server describes a part of that system, inclusive of roles, environments, cookbooks, and the other Chef Infra Server objects necessary to configure that part of the system.
Policyfile encourages safer workflows by making it easier to publish development versions of cookbooks to the Chef Infra Server without the risk of mutating the production versions and without requiring a complicated versioning scheme to work around cookbook mutability issues. Roles are mutable and those changes are applied only to the nodes specified by the policy. Policyfile does not require any changes to your normal workflows. Use the same repositories you are already using, the same cookbooks, and workflows. Policyfile will prevent an updated cookbook or role from being applied immediately to all machines.
When running Chef Infra without a Policyfile, the exact set of cookbooks that are applied to a node is defined by:
run_list propertyThese conditions are re-evaluated every time Chef Infra Client runs, which can make it harder to know which cookbooks will be run by Chef Infra Client or what the effects of updating a role or uploading a new cookbook will be.
Policyfile simplifies this behavior by computing the cookbook set on the workstation, and then producing a readable document of that solution: a Policyfile.lock.json file. This pre-computed file is uploaded to the Chef Infra Server, and is then used in each Chef Infra Client run that is managed by that particular policy name and policy group.
When running Chef Infra without Policyfile, the Chef Infra Server loads dependency data for all known versions of all known cookbooks, and then runs an expensive computation to determine the correct set.
Policyfile moves this computation to the workstation, where it is done less frequently.
When running Chef Infra without Policyfile roles and environments are global objects. Changes to roles and environments are applied immediately to any node that contains that role in its run-list or belong to a particular environment. This can make it hard to update roles and environments and in some use cases discourages using them at all.
Policyfile effectively replaces roles and environments. Policyfile files are versioned automatically and new versions are applied to systems only when promoted.
When running Chef without Policyfile, existing versions of cookbooks are mutable. This is convenient for many use cases, especially if users upload in-development cookbook revisions to the Chef Infra Server. But this sometimes creates issues that are similar to role mutability by allowing those cookbook changes to be applied immediately to nodes that use that cookbook. Users account for this by rigorous testing processes to ensure that only fully integrated cookbooks are ever published. This process does enforce good development habits, but at the same time it shouldn’t be a required part of a workflow that ends with publishing an in-development cookbook to the Chef Infra Server for testing against real nodes Policyfile solves this issue by using a cookbook publishing API for the Chef Infra Server that does not provide cookbook mutability. Name collisions are prevented by storing cookbooks by name and an opaque identifier that is computed from the content of the cookbook itself.
For example, name/version collisions can occur when users temporarily fork an upstream cookbook. Even if the user contributes their change and the maintainer is responsive, there may be a period of time where the user needs their fork in order to make progress. This situation presents a versioning dilemma: if the user doesn’t update their own version, they must overwrite the existing copy of that cookbook on the Chef Infra Server, wheres if they do update the version number it might conflict with the version number of the future release of that upstream cookbook.
The opaque identifier that is computed from the content of a cookbook is the only place where an opaque identifier is necessary. When working with Policyfile, be sure to:
Policyfile.rb fileExtra metadata about the cookbook is stored and included in Chef Infra Server API responses and in the Policyfile.lock.json file, including the source of a cookbook (Chef Supermarket, git, local disk, etc.), as well as any upstream identifiers, such as git revisions. For cookbooks that are loaded from the local disk that are in a git repo, the upstream URL, current revision ID, and the state of the repo are stored also.
The opaque identifier is mostly behind the scenes and is only visible once published to the Chef Infra Server. Cookbooks that are uploaded to the Chef Infra Server may have extended version numbers such as 1.0.0-dev.
Policyfile replaces the environment cookbook pattern that is often required by Berkshelf, along with a dependency solver and fetcher. That said, Policyfile does not replace all Berkshelf scenarios.
The following knife commands used to set the policy group and policy name on the Chef Infra Server. For example:
knife node policy set test-node 'test-policy-group-name' 'test-policy-name'
A Policyfile file allows you to specify in a single document the cookbook revisions and recipes that Chef Infra Client will apply. A Policyfile file is uploaded to the Chef Infra Server, where it is associated with a group of nodes. When these nodes are configured during a Chef Infra Client run, Chef Infra Client will make decisions based on your Policyfile settings and will build a run-list based on that information. A Policyfile file may be versioned, and then promoted through deployment stages to safely and reliably deploy new configuration.
A Policyfile.rb is a Ruby file in which run-list and cookbook locations are specified. The syntax is as follows:
name "name"
run_list "ITEM", "ITEM", ...
default_source :SOURCE_TYPE, *args
cookbook "NAME" [, "VERSION_CONSTRAINT"] [, SOURCE_OPTIONS]
A Policyfile.rb file may contain the following settings:
name "name"Required. The name of the policy. Use a name that reflects the purpose of the machines against which the policy will run.
run_list "ITEM", "ITEM", ...Required. The run-list Chef Infra Client will use to apply the policy to one (or more) nodes.
default_source :SOURCE_TYPE, *argsThe location in which any cookbooks not specified by cookbook are located. Possible values: chef_repo, chef_server, :supermarket, and :artifactory. Use more than one default_source to specify more than one location for cookbooks.
default_source :supermarket pulls cookbooks from the public Chef Supermarket.
default_source :supermarket, "https://mysupermarket.example" pulls cookbooks from a named private Chef Supermarket.
default_source :chef_server, "https://chef-server.example/organizations/example" pulls cookbooks from the Chef Infra Server.
default_source :chef_repo, "path/to/repo" pulls cookbooks from a monolithic cookbook repository. This may be a path to the top-level of a cookbook repository or to the /cookbooks directory within that repository.
default_source :artifactory, "https://artifactory.example/api/chef/my-supermarket" pulls cookbooks from an Artifactory server. Requires either artifactory_api_key to be set in config.rb or ARTIFACTORY_API_KEY to be set in your environment.
Multiple cookbook sources may be specified. For example from the public Chef Supermarket and a monolithic repository:
default_source :supermarket
default_source :chef_repo, 'path/to/repo'
or from both a public and private Chef Supermarket:
default_source :supermarket
default_source :supermarket, 'https://supermarket.example'
Note
If a run-list or any dependencies require a cookbook that is present in more than one source, be explicit about which source is preferred. This will ensure that a cookbook is always pulled from an expected source. For example, an internally-developed cookbook named chef-client will conflict with the public chef-client cookbook that is maintained by Chef. To specify a named source for a cookbook:
default_source :supermarket
default_source :supermarket, 'https://supermarket.example' do |s|
s.preferred_for 'chef-client'
end
List multiple cookbooks on the same line:
default_source :supermarket
default_source :supermarket, 'https://supermarket.example' do |s|
s.preferred_for 'chef-client', 'nginx', 'mysql'
end
cookbook "NAME" [, "VERSION_CONSTRAINT"] [, SOURCE_OPTIONS]Add cookbooks to the policy, specify a version constraint, or specify an alternate source location, such as Chef Supermarket. For example, add a cookbook:
cookbook 'apache2'
Specify a version constraint:
run_list 'jenkins::master'
# Restrict the jenkins cookbook to version 2.x, greater than 2.1
cookbook 'jenkins', '~> 2.1'
Specify an alternate source:
cookbook 'my_app', path: 'cookbooks/my_app'
or:
cookbook 'mysql', github: 'opscode-cookbooks/mysql', branch: 'master'
or:
cookbook 'chef-ingredient', git: 'https://github.com/chef-cookbooks/chef-ingredient.git', tag: 'v0.12.0'
named_run_list "NAME", "ITEM1", "ITEM2", ...Specify a named run-list to be used as an alternative to the override run-list. This setting should be used carefully and for specific use cases, like running a small set of recipes to quickly converge configuration for a single application on a host or for one-time setup tasks. For example:
named_run_list :update_app, 'my_app_cookbook::default'
include_policy "NAME", *argsSpecify a policyfile lock to be merged with this policy. Chef Workstation supports pulling this lock from a local or remote file, from a Chef Infra Server, or from a git repository. When the policyfile lock is included, its run-list will appear before the current policyfile’s run-list. This setting requires that the solved cookbooks appear as-is from the included policyfile lock. If conflicting attributes or cookbooks are provided, an error will be presented. See RFC097 for the full specifications of this feature.
Pull the policyfile lock from ./NAME.lock.json:
include_policy 'NAME', path: '.'
Pull the policyfile lock from ./foo.lock.json.
include_policy 'NAME', path: './foo.lock.json'
Pull the policyfile lock foo.lock.json from the example/foo Git repository on the git.example.com Git server.
include_policy 'NAME', git: 'https://git.example.com/example/foo', path: 'foo.lock.json'
Pull the policyfile lock from ./bar.lock.json with revision ID ‘revision1’.
include_policy 'NAME', policy_revision_id: 'revision1', path: './bar.lock.json'
Pull the policyfile lock from a remote server https://internal.example.com/foo.lock.json.
include_policy 'NAME', remote: 'https://internal.example.com/foo.lock.json'
Pull the policyfile lock from a remote server https://internal.example.com/bar.lock.json and with revision ID ‘revision1’.
include_policy 'NAME', policy_revision_id: 'revision1', remote: 'https://internal.example.com/foo.lock.json'
Pull the policy NAME with revision ID revision1 from the http://chef-server.example Chef Infra Server:
include_policy 'NAME', policy_revision_id: 'revision1', server: 'http://chef-server.example'
Pull the policy foo with revision ID revision1:
include_policy 'NAME', policy_name: 'foo', policy_revision_id: 'revision1', server: 'http://chef-server.example'
Pull and lock the current revision for policy foo in policy group prod:
include_policy 'NAME', policy_name: 'foo', policy_group: 'prod', server: 'http://chef-server.example'
For example:
name 'jenkins-master'
run_list 'java', 'jenkins::master', 'recipe[policyfile_demo]'
default_source :supermarket, 'https://mysupermarket.example'
cookbook 'policyfile_demo', path: 'cookbooks/policyfile_demo'
cookbook 'jenkins', '~> 8.2'
cookbook 'mysql', github: 'sous-chefs/mysql', branch: 'master'
The following settings may be configured in the client.rb file for use with Policyfile:
named_run_listThe run-list associated with a policy file.
policy_groupThe name of a policy group that exists on the Chef Infra Server. policy_name must also be specified.
policy_nameThe name of a policy, as identified by the name setting in a Policyfile.rb file. policy_group must also be specified.
use_policyfileChef Infra Client automatically checks the configuration, node JSON, and the stored node on the Chef Infra Server to determine if Policyfile files are being used, and then automatically updates this flag. Default value: false.
A node may be bootstrapped to use Policyfile files. Use the following options as part of the bootstrap command:
--policy-group POLICY_GROUPThe name of a policy group that exists on the Chef Infra Server.
--policy-name POLICY_NAMEThe name of a policy, as identified by the name setting in a Policyfile.rb file.
For a customized bootstrap process, add policy_name and policy_group to the first-boot JSON file that is passed to Chef Infra Client.
The policy_name and policy_group settings for a node are stored as searchable attributes and as such are available when using a fuzzy matching search pattern. For example: knife search dev will return nodes that are part of the dev policy group.
Kitchen may be used to test Policyfile files. Add the following to kitchen.yml:
provisioner:name:chef_zeroA named run-list may be used on a per-suite basis:
suites:- name:clientprovisioner:named_run_list:test_client_recipe- name:serverprovisioner:named_run_list:test_server_recipeor globally:
provisioner:name:chef_zeronamed_run_list:integration_test_run_listor testing with policies per-suite, once the Policyfile files are available in your repo:
suites:- name:defaultmegaprovisioner:policyfile:policies/default.rb- name:defaultultraprovisioner:policyfile:policies/defaulttwo.rbNote
chef_zero explicitly tests outside the context of a Chef Infra Server, the policy_groups concept is not applicable. The value of policy_group during a converge will be set to local. The following commands are built into the chef executable and support the use of Policyfile files.
Use the chef clean-policy-cookbooks subcommand to delete cookbooks that are not used by Policyfile files. Cookbooks are considered unused when they are not referenced by any policy revisions on the Chef Infra Server.
Note
Cookbooks that are referenced by orphaned policy revisions are not removed. Use chef clean-policy-revisions to remove orphaned policies.
This subcommand has the following syntax:
chef clean-policy-cookbooks (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef clean-policy-revisions subcommand to delete orphaned policy revisions to Policyfile files from the Chef Infra Server. An orphaned policy revision is not associated to any policy group and therefore is not in active use by any node. Use chef show-policy --orphans to view a list of orphaned policy revisions.
This subcommand has the following syntax:
chef clean-policy-revisions (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef delete-policy subcommand to delete all revisions of the named policy that exist on the Chef Infra Server. (The state of the policy revision is backed up locally and may be restored using the chef undelete subcommand.)
This subcommand has the following syntax:
chef delete-policy POLICY_NAME (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef delete-policy-group subcommand to delete the named policy group from the Chef Infra Server. Any policy revision associated with that policy group is not deleted. (The state of the policy group is backed up locally and may be restored using the chef undelete subcommand.)
This subcommand has the following syntax:
chef delete-policy-group POLICY_GROUP (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef diff subcommand to display an itemized comparison of two revisions of a Policyfile.lock.json file.
This subcommand has the following syntax:
chef diff POLICY_FILE --head | --git POLICY_GROUP | POLICY_GROUP...POLICY_GROUP (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-g GIT_REF, --git GIT_REF
Compare the specified git reference against the current revision of a Policyfile.lock.json file or against another git reference.
-h, --help
Show help for the command.
--headA shortcut for chef diff --git HEAD. When a git-specific flag is not provided, the on-disk Policyfile.lock.json file is compared to one on the Chef Infra Server or (if a Policyfile.lock.json file is not present on-disk) two Policyfile.lock.json files in the specified policy group on the Chef Infra Server are compared.
--[no-]pagerUse --pager to enable paged output for a Policyfile.lock.json file. Default value: --pager.
-v, --version
The Chef Infra Client version.
chef diff --git HEAD
chef diff --git master
chef diff --git v1.0.0
chef diff --git master...dev
chef diff --git v1.0.0...master
chef diff staging
chef diff production...staging
Use the chef export subcommand to create a chef-zero-compatible chef-repo that contains the cookbooks described by a Policyfile.lock.json file. After a chef-zero-compatible chef-repo is copied to a node, the policy can be applied locally on that machine by running chef-client -z (local mode).
This subcommand has the following syntax:
chef export POLICY_FILE DIRECTORY (options)
This subcommand has the following options:
-a, --archive
Export an archive as a tarball, instead as a directory. Default value: false.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-f, --force
Remove the contents of the destination directory if that directory is not empty. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef generate policyfile subcommand to generate a file to be used with Policyfile.
This subcommand has the following syntax:
chef generate policyfile POLICY_NAME (options)
This subcommand has the following options:
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef generate repo subcommand to create a chef-repo. By default, the repo is a cookbook repo with options available to support generating a cookbook that supports Policyfile.
Note
This subcommand has the following syntax:
chef generate repo REPO_NAME (options)
This subcommand has the following options:
-h, --help
Show help for the command.
-p, --policy-only
Create a repository that does not store cookbook files, only Policyfile files.
-P, --policy
Use Policyfile instead of Berkshelf.
-r, --roles
Create directories for /roles and /environments instead of creating directories for Policyfile.
-v, --version
The Chef Infra Client version.
Use the chef install subcommand to evaluate a policy file and find a compatible set of cookbooks, build a run-list, cache it locally, and then emit a Policyfile.lock.json file that describes the locked policy set. The Policyfile.lock.json file may be used to install the locked policy set to other machines and may be pushed to a policy group on the Chef Infra Server to apply that policy to a group of nodes that are under management by Chef.
This subcommand has the following syntax:
chef install POLICY_FILE (options)
This subcommand has the following options:
-D, --debug
Enable stack traces and other debug output.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
When the chef install command is run, Chef Workstation caches any necessary cookbooks and emits a Policyfile.lock.json file that describes:
A Policyfile.lock.json file is associated with a specific policy group, i.e. is associated with one (or more) nodes that use the same revision of a given policy.
A Policyfile.lock.json file is similar to:
{
"revision_id": "288ed244f8db8bff3caf58147e840bbe079f76e0",
"name": "jenkins",
"run_list": [
"recipe[java::default]",
"recipe[jenkins::master]",
"recipe[policyfile_demo::default]"
],
"cookbook_locks": {
"policyfile_demo": {
"version": "0.1.0",
"identifier": "f04cc40faf628253fe7d9566d66a1733fb1afbe9",
"dotted_decimal_identifier": "67638399371010690.23642238397896298.25512023620585",
"source": "cookbooks/policyfile_demo",
"cache_key": null,
"scm_info": null,
"source_options": {
"path": "cookbooks/policyfile_demo"
}
},
"java": {
"version": "1.24.0",
"identifier": "4c24ae46a6633e424925c24e683e0f43786236a3",
"dotted_decimal_identifier": "21432429158228798.18657774985439294.16782456927907",
"cache_key": "java-1.24.0-supermarket.chef.io",
"origin": "https://supermarket.chef.io/api/v1/cookbooks/java/versions/1.24.0/download",
"source_options": {
"artifactserver": "https://supermarket.chef.io/api/v1/cookbooks/java/versions/1.24.0/download",
"version": "1.24.0"
}
Use the chef push subcommand to upload an existing Policyfile.lock.json file to the Chef Infra Server, along with all of the cookbooks that are contained in the file. The Policyfile.lock.json file will be applied to the specified policy group, which is a set of nodes that share the same run-list and cookbooks.
This subcommand has the following syntax:
chef push POLICY_GROUP POLICY_FILE (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
The chef push-archive subcommand is used to publish a policy archive file to the Chef Infra Server. (A policy archive is created using the chef export subcommand.) The policy archive is assigned to the specified policy group, which is a set of nodes that share the same run-list and cookbooks.
This subcommand has the following syntax:
chef push-archive POLICY_GROUP ARCHIVE_FILE (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
Use the chef show-policy subcommand to display revisions for every Policyfile.rb file that is on the Chef Infra Server. By default, only active policy revisions are shown. When both a policy and policy group are specified, the contents of the active Policyfile.lock.json file for the policy group is returned.
This subcommand has the following syntax:
chef show-policy POLICY_NAME POLICY_GROUP (options)
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-o, --orphans
Show policy revisions that are not currently assigned to any policy group.
--[no-]pagerUse --pager to enable paged output for a Policyfile.lock.json file. Default value: --pager.
-v, --version
The Chef Infra Client version.
Use the chef undelete subcommand to recover a deleted policy or policy group. This command:
This subcommand has the following syntax:
chef undelete (options)
When run with no arguments, returns a list of available operations.
This subcommand has the following options:
-c CONFIG_FILE, --config CONFIG_FILE
The path to the knife configuration file.
-D, --debug
Enable stack traces and other debug output.
-h, --help
Show help for the command.
-i ID, --id ID
Undo the delete operation specified by ID.
-l, --last
Undo the most recent delete operation.
--listDefault. Return a list of available operations.
-v, --version
The Chef Infra Client version.
Use the chef update subcommand to read the Policyfile.rb file, and then apply any changes. This will resolve dependencies and will create a Policyfile.lock.json file. The locked policy will reflect any changes to the run-list and will pull in any cookbook updates that are compatible with any version constraints defined in the Policyfile.rb file.
This subcommand has the following syntax:
chef update POLICY_FILE (options)
This subcommand has the following options:
-a, --attributes
Update attributes. Default value: false.
-D, --debug
Enable stack traces and other debug output. Default value: false.
-h, --help
Show help for the command.
-v, --version
The Chef Infra Client version.
© Chef Software, Inc.
Licensed under the Creative Commons Attribution 3.0 Unported License.
The Chef™ Mark and Chef Logo are either registered trademarks/service marks or trademarks/servicemarks of Chef, in the United States and other countries and are used with Chef Inc's permission.
We are not affiliated with, endorsed or sponsored by Chef Inc.
https://docs.chef.io/policyfile/