W3cubDocs

/WordPress

do_robots()

Displays the default robots.txt file content.

Source

File: wp-includes/functions.php 

function do_robots() {
	header( 'Content-Type: text/plain; charset=utf-8' );

	/**
	 * Fires when displaying the robots.txt file.
	 *
	 * @since 2.1.0
	 */
	do_action( 'do_robotstxt' );

	$output = "User-agent: *\n";
	$public = get_option( 'blog_public' );

	$site_url = parse_url( site_url() );
	$path     = ( ! empty( $site_url['path'] ) ) ? $site_url['path'] : '';
	$output  .= "Disallow: $path/wp-admin/\n";
	$output  .= "Allow: $path/wp-admin/admin-ajax.php\n";

	/**
	 * Filters the robots.txt output.
	 *
	 * @since 3.0.0
	 *
	 * @param string $output The robots.txt output.
	 * @param bool   $public Whether the site is considered "public".
	 */
	echo apply_filters( 'robots_txt', $output, $public );
}

Uses

Uses Description
wp-includes/functions.php: do_robotstxt

Fires when displaying the robots.txt file.
wp-includes/functions.php: robots_txt

Filters the robots.txt output.
wp-includes/link-template.php: site_url()

Retrieves the URL for the current site where WordPress application files (e.g. wp-blog-header.php or the wp-admin/ folder) are accessible.
wp-includes/plugin.php: do_action()

Execute functions hooked on a specific action hook.
wp-includes/plugin.php: apply_filters()

Calls the callback functions that have been added to a filter hook.
wp-includes/option.php: get_option()

Retrieves an option value based on an option name.

Changelog

Version Description
5.3.0 Remove the "Disallow: /" output if search engine visiblity is discouraged in favor of robots meta HTML tag in wp_no_robots().
2.1.0 Introduced.

© 2003–2019 WordPress Foundation
Licensed under the GNU GPLv2+ License.
https://developer.wordpress.org/reference/functions/do_robots