7 * When a drush command is called with the --backend option,
8 * it will buffer all output, and instead return a JSON encoded
9 * string containing all relevant information on the command that
12 * Through this mechanism, it is possible for Drush commands to
15 * There are many cases where a command might wish to call another
16 * command in its own process, to allow the calling command to
17 * intercept and act on any errors that may occur in the script that
20 * A simple example is if there exists an 'update' command for running
21 * update.php on a specific site. The original command might download
22 * a newer version of a module for installation on a site, and then
23 * run the update script in a separate process, so that in the case
24 * of an error running a hook_update_n function, the module can revert
25 * to a previously made database backup, and the previously installed code.
27 * By calling the script in a separate process, the calling script is insulated
28 * from any error that occurs in the called script, to the level that if a
29 * php code error occurs (ie: misformed file, missing parenthesis, whatever),
30 * it is still able to reliably handle any problems that occur.
32 * This is nearly a RESTful API. @see http://en.wikipedia.org/wiki/REST
35 * http://[server]/[apipath]/[command]?[arg1]=[value1],[arg2]=[value2]
38 * [apipath] [command] --[arg1]=[value1] --[arg2]=[value2] --backend
40 * [apipath] in this case will be the path to the drush.php file.
41 * [command] is the command you would call, for instance 'status'.
43 * GET parameters will be passed as options to the script.
44 * POST parameters will be passed to the script as a JSON encoded associative array over STDIN.
46 * Because of this standard interface, Drush commands can also be executed on
47 * external servers through SSH pipes, simply by prepending, 'ssh username@server.com'
48 * in front of the command.
50 * If the key-based ssh authentication has been set up between the servers,
51 * this will just work. By default, drush is configured to disallow password
52 * authentication; if you would like to enter a password for every connection,
53 * then in your drushrc.php file, set $options['ssh-options'] so that it does NOT
54 * include '-o PasswordAuthentication=no'. See examples/example.drushrc.php.
56 * The results from backend API calls can be fetched via a call to
57 * drush_backend_get_result().
60 use Drush\Log\LogLevel;
61 use Drush\Preflight\PreflightArgs;
64 * Identify the JSON encoded output from a command.
66 * Note that Drush now outputs a null ("\0") before the DRUSH_BACKEND_OUTPUT_DELIMITER,
67 * but this null occurs where this constant is output rather than being
68 * included in the define. This is done to maintain compatibility with
69 * older versions of Drush, so that Drush-7.x can correctly parse backend messages
70 * from calls made to Drush-5.x and earlier. The null is removed via trim().
72 define('DRUSH_BACKEND_OUTPUT_START', 'DRUSH_BACKEND_OUTPUT_START>>>');
73 define('DRUSH_BACKEND_OUTPUT_DELIMITER', DRUSH_BACKEND_OUTPUT_START . '%s<<<DRUSH_BACKEND_OUTPUT_END');
76 * Identify JSON encoded "packets" embedded inside of backend
77 * output; used to send out-of-band information durring a backend
78 * invoke call (currently only used for log and error messages).
80 define('DRUSH_BACKEND_PACKET_START', "DRUSH_BACKEND:");
81 define('DRUSH_BACKEND_PACKET_PATTERN', "\0" . DRUSH_BACKEND_PACKET_START . "%s\n\0");
84 * The backend result is the original PHP data structure (usually an array)
85 * used to generate the output for the current command.
87 function drush_backend_set_result($value) {
88 if (\Drush\Drush::backend()) {
89 drush_set_context('BACKEND_RESULT', $value);
94 * Retrieves the results from the last call to backend_invoke.
97 * An associative array containing information from the last
98 * backend invoke. The keys in the array include:
100 * - output: This item contains the textual output of
101 * the command that was executed.
102 * - object: Contains the PHP object representation of the
103 * result of the command.
104 * - self: The self object contains the alias record that was
105 * used to select the bootstrapped site when the command was
107 * - error_status: This item returns the error status for the
108 * command. Zero means "no error".
109 * - log: The log item contains an array of log messages from
110 * the command execution ordered chronologically. Each log
111 * entry is an associative array. A log entry contains
113 * o type: The type of log entry, such as 'notice' or 'warning'
114 * o message: The log message
115 * o timestamp: The time that the message was logged
116 * o memory: Available memory at the time that the message was logged
117 * o error: The error code associated with the log message
118 * (only for log entries whose type is 'error')
119 * - error_log: The error_log item contains another representation
120 * of entries from the log. Only log entries whose 'error' item
121 * is set will appear in the error log. The error log is an
122 * associative array whose key is the error code, and whose value
123 * is an array of messages--one message for every log entry with
124 * the same error code.
125 * - context: The context item contains a representation of all option
126 * values that affected the operation of the command, including both
127 * the command line options, options set in a drushrc.php configuration
128 * files, and options set from the alias record used with the command.
130 function drush_backend_get_result() {
131 return drush_get_context('BACKEND_RESULT');
135 * Print the json-encoded output of this command, including the
136 * encoded log records, context information, etc.
138 function drush_backend_output() {
141 // Strip out backend commands.
142 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), ["\0" => "\\0"]);
143 $packet_regex = str_replace("\n", "", $packet_regex);
144 $data['output'] = preg_replace("/$packet_regex/s", '', drush_backend_output_collect(NULL));
146 if (drush_get_context('DRUSH_QUIET', FALSE)) {
150 $result_object = drush_backend_get_result();
151 if (isset($result_object)) {
152 $data['object'] = $result_object;
155 $error = drush_get_error();
156 $data['error_status'] = ($error) ? $error : DRUSH_SUCCESS;
158 $data['log'] = drush_get_log(); // Append logging information
159 // The error log is a more specific version of the log, and may be used by calling
160 // scripts to check for specific errors that have occurred.
161 $data['error_log'] = drush_get_error_log();
162 // If there is a @self record, then include it in the result
163 $self_record = drush_sitealias_get_record('@self');
164 if (!empty($self_record)) {
165 $site_context = drush_get_context('site', []);
166 unset($site_context['config-file']);
167 unset($site_context['context-path']);
168 unset($self_record['loaded-config']);
169 unset($self_record['#name']);
170 $data['self'] = array_merge($site_context, $self_record);
173 // Return the options that were set at the end of the process.
174 $data['context'] = drush_get_merged_options();
175 printf("\0" . DRUSH_BACKEND_OUTPUT_DELIMITER, json_encode($data));
179 * Callback to collect backend command output.
181 function drush_backend_output_collect($string) {
183 if (!isset($string)) {
192 * Output buffer functions that discards all output but backend packets.
194 function drush_backend_output_discard($string) {
195 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), ["\0" => "\\0"]);
196 $packet_regex = str_replace("\n", "", $packet_regex);
197 if (preg_match_all("/$packet_regex/s", $string, $matches)) {
198 return implode('', $matches[0]);
203 * Output a backend packet if we're running as backend.
206 * The packet to send.
208 * Data for the command.
211 * A boolean indicating whether the command was output.
213 function drush_backend_packet($packet, $data) {
214 if (\Drush\Drush::backend()) {
215 $data['packet'] = $packet;
216 $data = json_encode($data);
217 // We use 'fwrite' instead of 'drush_print' here because
218 // this backend packet is out-of-band data.
219 fwrite(STDERR, sprintf(DRUSH_BACKEND_PACKET_PATTERN, $data));
227 * Parse output returned from a Drush command.
230 * The output of a drush command
232 * Integrate the errors and log messages from the command into the current process.
234 * Whether output has already been handled.
237 * An associative array containing the data from the external command, or the string parameter if it
238 * could not be parsed successfully.
240 function drush_backend_parse_output($string, $backend_options = [], $outputted = FALSE) {
241 $regex = sprintf(DRUSH_BACKEND_OUTPUT_DELIMITER, '(.*)');
243 preg_match("/$regex/s", $string, $match);
245 if (!empty($match) && $match[1]) {
246 // we have our JSON encoded string
248 // remove the match we just made and any non printing characters
249 $string = trim(str_replace(sprintf(DRUSH_BACKEND_OUTPUT_DELIMITER, $match[1]), '', $string));
252 if (!empty($output)) {
253 $data = json_decode($output, TRUE);
254 if (is_array($data)) {
255 _drush_backend_integrate($data, $backend_options, $outputted);
263 * Integrate log messages and error statuses into the current
266 * Output produced by the called script will be printed if we didn't print it
267 * on the fly, errors will be set, and log messages will be logged locally, if
268 * not already logged.
271 * The associative array returned from the external command.
273 * Whether output has already been handled.
275 function _drush_backend_integrate($data, $backend_options, $outputted) {
276 // In 'integrate' mode, logs and errors have already been handled
277 // by drush_backend_packet (sender) drush_backend_parse_packets (receiver - us)
278 // during incremental output. We therefore do not need to call drush_set_error
279 // or drush_log here. The exception is if the sender is an older version of
280 // Drush (version 4.x) that does not send backend packets, then we will
281 // not have processed the log entries yet, and must print them here.
282 $received_packets = drush_get_context('DRUSH_RECEIVED_BACKEND_PACKETS', FALSE);
283 if (is_array($data['log']) && $backend_options['log'] && (!$received_packets)) {
284 foreach($data['log'] as $log) {
285 $message = is_array($log['message']) ? implode("\n", $log['message']) : $log['message'];
286 if (isset($backend_options['#output-label'])) {
287 $message = $backend_options['#output-label'] . $message;
289 if (isset($log['error']) && $backend_options['integrate']) {
290 drush_set_error($log['error'], $message);
292 elseif ($backend_options['integrate']) {
293 drush_log($message, $log['type']);
297 // Output will either be printed, or buffered to the drush_backend_output command.
298 // If the output has already been printed, then we do not need to show it again on a failure.
300 if (drush_cmp_error('DRUSH_APPLICATION_ERROR') && !empty($data['output'])) {
301 drush_set_error("DRUSH_APPLICATION_ERROR", dt("Output from failed command :\n !output", ['!output' => $data['output']]));
303 elseif ($backend_options['output']) {
304 _drush_backend_print_output($data['output'], $backend_options);
310 * Supress log message output during backend integrate.
312 function _drush_backend_integrate_log($entry) {
316 * Call an external command using proc_open.
319 * An array of records containing the following elements:
320 * 'cmd' - The command to execute, already properly escaped
321 * 'post-options' - An associative array that will be JSON encoded
322 * and passed to the script being called. Objects are not allowed,
323 * as they do not json_decode gracefully.
324 * 'backend-options' - Options that control the operation of the backend invoke
326 * An array of commands to execute. These commands already need to be properly escaped.
327 * In this case, post-options will default to empty, and a default output label will
330 * An associative array that will be JSON encoded and passed to the script being called.
331 * Objects are not allowed, as they do not json_decode gracefully.
334 * False if the command could not be executed, or did not return any output.
335 * If it executed successfully, it returns an associative array containing the command
336 * called, the output of the command, and the error code of the command.
338 function _drush_backend_proc_open($cmds, $process_limit, $context = NULL) {
340 0 => ["pipe", "r"], // stdin is a pipe that the child will read from
341 1 => ["pipe", "w"], // stdout is a pipe that the child will write to
344 $open_processes = [];
346 $process_limit = max($process_limit, 1);
347 $is_windows = drush_is_windows();
348 // Loop through processes until they all close, having a nap as needed.
350 while (count($open_processes) || count($cmds)) {
352 if (count($cmds) && (count($open_processes) < $process_limit)) {
353 // Pop the site and command (key / value) from the cmds array
355 $cmd = current($cmds);
359 if (is_array($cmd)) {
361 $post_options = $cmd['post-options'];
362 $backend_options = $cmd['backend-options'];
367 $backend_options = [];
369 $backend_options += [
370 '#output-label' => '',
371 '#process-read-size' => 4096,
374 drush_log($backend_options['#output-label'] . $c);
375 $process['process'] = proc_open($c, $descriptorspec, $process['pipes'], null, null, ['context' => $context]);
376 if (is_resource($process['process'])) {
378 fwrite($process['pipes'][0], json_encode($post_options)); // pass the data array in a JSON encoded string
380 // If we do not close stdin here, then we cause a deadlock;
381 // see: http://drupal.org/node/766080#comment-4309936
382 // If we reimplement interactive commands to also use
383 // _drush_proc_open, then clearly we would need to keep
385 fclose($process['pipes'][0]);
387 $process['info'] = stream_get_meta_data($process['pipes'][1]);
388 stream_set_blocking($process['pipes'][1], FALSE);
389 stream_set_timeout($process['pipes'][1], 1);
390 $bucket[$site]['cmd'] = $c;
391 $bucket[$site]['output'] = '';
392 $bucket[$site]['remainder'] = '';
393 $bucket[$site]['backend-options'] = $backend_options;
394 $bucket[$site]['end_of_output'] = FALSE;
395 $bucket[$site]['outputted'] = FALSE;
396 $open_processes[$site] = $process;
398 // Reset the $nap_time variable as there might be output to process next
402 // Set up to call stream_select(). See:
403 // http://php.net/manual/en/function.stream-select.php
404 // We can't use stream_select on Windows, because it doesn't work for
405 // streams returned by proc_open.
410 $except_streams = [];
411 foreach ($open_processes as $site => &$current_process) {
412 if (isset($current_process['pipes'][1])) {
413 $read_streams[] = $current_process['pipes'][1];
416 // Wait up to 2s for data to become ready on one of the read streams.
417 if (count($read_streams)) {
418 $ss_result = stream_select($read_streams, $write_streams, $except_streams, 2);
419 // If stream_select returns a error, then fallback to using $nap_time.
420 if ($ss_result !== FALSE) {
426 foreach ($open_processes as $site => &$current_process) {
427 if (isset($current_process['pipes'][1])) {
428 // Collect output from stdout
429 $bucket[$site][1] = '';
430 $info = stream_get_meta_data($current_process['pipes'][1]);
432 if (!feof($current_process['pipes'][1]) && !$info['timed_out']) {
433 $string = $bucket[$site]['remainder'] . fread($current_process['pipes'][1], $backend_options['#process-read-size']);
434 $bucket[$site]['remainder'] = '';
435 $output_end_pos = strpos($string, DRUSH_BACKEND_OUTPUT_START);
436 if ($output_end_pos !== FALSE) {
437 $trailing_string = substr($string, 0, $output_end_pos);
438 $trailing_remainder = '';
439 // If there is any data in the trailing string (characters prior
440 // to the backend output start), then process any backend packets
442 if (strlen($trailing_string) > 0) {
443 drush_backend_parse_packets($trailing_string, $trailing_remainder, $bucket[$site]['backend-options']);
445 // If there is any data remaining in the trailing string after
446 // the backend packets are removed, then print it.
447 if (strlen($trailing_string) > 0) {
448 _drush_backend_print_output($trailing_string . $trailing_remainder, $bucket[$site]['backend-options']);
449 $bucket[$site]['outputted'] = TRUE;
451 $bucket[$site]['end_of_output'] = TRUE;
453 if (!$bucket[$site]['end_of_output']) {
454 drush_backend_parse_packets($string, $bucket[$site]['remainder'], $bucket[$site]['backend-options']);
455 // Pass output through.
456 _drush_backend_print_output($string, $bucket[$site]['backend-options']);
457 if (strlen($string) > 0) {
458 $bucket[$site]['outputted'] = TRUE;
461 $bucket[$site][1] .= $string;
462 $bucket[$site]['output'] .= $string;
463 $info = stream_get_meta_data($current_process['pipes'][1]);
466 // Reset the $nap_time variable as there might be output to process
468 if (strlen($string) > 0) {
473 fclose($current_process['pipes'][1]);
474 unset($current_process['pipes'][1]);
475 // close the pipe , set a marker
477 // Reset the $nap_time variable as there might be output to process
483 // if both pipes are closed for the process, remove it from active loop and add a new process to open.
484 $bucket[$site]['code'] = proc_close($current_process['process']);
485 unset($open_processes[$site]);
487 // Reset the $nap_time variable as there might be output to process next
493 // We should sleep for a bit if we need to, up to a maximum of 1/10 of a
496 usleep(max($nap_time * 500, 100000));
500 // TODO: Handle bad proc handles
508 * Print the output received from a call to backend invoke,
509 * adding the label to the head of each line if necessary.
511 function _drush_backend_print_output($output_string, $backend_options) {
512 if ($backend_options['output'] && !empty($output_string)) {
513 $output_label = array_key_exists('#output-label', $backend_options) ? $backend_options['#output-label'] : FALSE;
514 if (!empty($output_label)) {
515 // Remove one, and only one newline from the end of the
516 // string. Else we'll get an extra 'empty' line.
517 foreach (explode("\n", preg_replace('/\\n$/', '', $output_string)) as $line) {
518 fwrite(STDOUT, $output_label . rtrim($line) . "\n");
522 fwrite(STDOUT, $output_string);
528 * Parse out and remove backend packet from the supplied string and
529 * invoke the commands.
531 function drush_backend_parse_packets(&$string, &$remainder, $backend_options) {
533 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), ["\0" => "\\0"]);
534 $packet_regex = str_replace("\n", "", $packet_regex);
535 if (preg_match_all("/$packet_regex/s", $string, $match, PREG_PATTERN_ORDER)) {
536 drush_set_context('DRUSH_RECEIVED_BACKEND_PACKETS', TRUE);
537 foreach ($match[1] as $packet_data) {
538 $entry = (array) json_decode($packet_data);
539 if (is_array($entry) && isset($entry['packet'])) {
540 $function = 'drush_backend_packet_' . $entry['packet'];
541 if (function_exists($function)) {
542 $function($entry, $backend_options);
545 drush_log(dt("Unknown backend packet @packet", ['@packet' => $entry['packet']]), LogLevel::INFO);
549 drush_log(dt("Malformed backend packet"), LogLevel::ERROR);
550 drush_log(dt("Bad packet: @packet", ['@packet' => print_r($entry, TRUE)]), LogLevel::DEBUG);
551 drush_log(dt("String is: @str", ['@str' => $packet_data]), LogLevel::DEBUG);
554 $string = preg_replace("/$packet_regex/s", '', $string);
556 // Check to see if there is potentially a partial packet remaining.
557 // We only care about the last null; if there are any nulls prior
558 // to the last one, they would have been removed above if they were
559 // valid drush packets.
560 $embedded_null = strrpos($string, "\0");
561 if ($embedded_null !== FALSE) {
562 // We will consider everything after $embedded_null to be part of
563 // the $remainder string if:
564 // - the embedded null is less than strlen(DRUSH_BACKEND_OUTPUT_START)
565 // from the end of $string (that is, there might be a truncated
566 // backend packet header, or the truncated backend output start
569 // - the embedded null is followed by DRUSH_BACKEND_PACKET_START
570 // (that is, the terminating null for that packet has not been
571 // read into our buffer yet)
572 if (($embedded_null + strlen(DRUSH_BACKEND_OUTPUT_START) >= strlen($string)) || (substr($string, $embedded_null + 1, strlen(DRUSH_BACKEND_PACKET_START)) == DRUSH_BACKEND_PACKET_START)) {
573 $remainder = substr($string, $embedded_null);
574 $string = substr($string, 0, $embedded_null);
580 * Backend command for setting errors.
582 function drush_backend_packet_set_error($data, $backend_options) {
583 if (!$backend_options['integrate']) {
587 if (array_key_exists('#output-label', $backend_options)) {
588 $output_label = $backend_options['#output-label'];
590 drush_set_error($data['error'], $data['message'], $output_label);
594 * Default options for backend_invoke commands.
596 function _drush_backend_adjust_options($site_record, $command, $command_options, $backend_options) {
597 // By default, if the caller does not specify a value for 'output', but does
598 // specify 'integrate' === FALSE, then we will set output to FALSE. Otherwise we
599 // will allow it to default to TRUE.
600 if ((array_key_exists('integrate', $backend_options)) && ($backend_options['integrate'] === FALSE) && (!array_key_exists('output', $backend_options))) {
601 $backend_options['output'] = FALSE;
603 $has_site_specification = array_key_exists('root', $site_record) || array_key_exists('uri', $site_record);
604 $result = $backend_options + [
610 'dispatch-using-alias' => !$has_site_specification,
612 // Convert '#integrate' et. al. into backend options
613 foreach ($command_options as $key => $value) {
614 if (substr($key,0,1) === '#') {
615 $result[substr($key,1)] = $value;
622 * Execute a new local or remote command in a new process.
624 * @deprecated as of Drush 9.4.0 and will be removed in Drush 10. Instead, use
625 * drush_invoke_process().
628 * An array of command records to execute. Each record should contain:
630 * An array containing information used to generate the command.
632 * Optional. A remote host to execute the drush command on.
634 * Optional. Defaults to the current user. If you specify this, you can choose which module to send.
636 * Optional. Defaults to "-o PasswordAuthentication=no"
638 * Optional. An associative array of environmental variables to prefix the Drush command with.
640 * Optional; contains paths to folders and executables useful to the command.
642 * Optional. Defaults to the current drush.php file on the local machine, and
643 * to simply 'drush' (the drush script in the current PATH) on remote servers.
644 * You may also specify a different drush.php script explicitly. You will need
645 * to set this when calling drush on a remote server if 'drush' is not in the
646 * PATH on that machine.
648 * A defined drush command such as 'cron', 'status' or any of the available ones such as 'drush pm'.
650 * An array of arguments for the command.
652 * Optional. An array containing options to pass to the remote script.
653 * Array items with a numeric key are treated as optional arguments to the
656 * Optional. Additional parameters that control the operation of the invoke.
658 * Optional. Defaults to 'GET'.
659 * If this parameter is set to 'POST', the $data array will be passed
660 * to the script being called as a JSON encoded string over the STDIN
661 * pipe of that process. This is preferable if you have to pass
662 * sensitive data such as passwords and the like.
663 * For any other value, the $data array will be collapsed down into a
664 * set of command line options to the script.
666 * Optional. Defaults to TRUE.
667 * If TRUE, any error statuses will be integrated into the current
668 * process. This might not be what you want, if you are writing a
669 * command that operates on multiple sites.
671 * Optional. Defaults to TRUE.
672 * If TRUE, any log messages will be integrated into the current
675 * Optional. Defaults to TRUE.
676 * If TRUE, output from the command will be synchronously printed to
679 * Optional. Defaults to the current drush.php file on the local
680 * machine, and to simply 'drush' (the drush script in the current
681 * PATH) on remote servers. You may also specify a different drush.php
682 * script explicitly. You will need to set this when calling drush on
683 * a remote server if 'drush' is not in the PATH on that machine.
684 * 'dispatch-using-alias'
685 * Optional. Defaults to FALSE.
686 * If specified as a non-empty value the drush command will be
687 * dispatched using the alias name on the command line, instead of
688 * the options from the alias being added to the command line
690 * @param common_options
691 * Optional. Merged in with the options for each invocation.
692 * @param backend_options
693 * Optional. Merged in with the backend options for each invocation.
694 * @param default_command
695 * Optional. Used as the 'command' for any invocation that does not
696 * define a command explicitly.
697 * @param default_site
698 * Optional. Used as the 'site' for any invocation that does not
699 * define a site explicitly.
701 * Optional. Passed in to proc_open if provided.
704 * If the command could not be completed successfully, FALSE.
705 * If the command was completed, this will return an associative array containing the data from drush_backend_output().
707 function drush_backend_invoke_concurrent($invocations, $common_options = [], $common_backend_options = [], $default_command = NULL, $default_site = NULL, $context = NULL) {
710 // Slice and dice our options in preparation to build a command string
711 $invocation_options = [];
712 foreach ($invocations as $invocation) {
713 $site_record = isset($invocation['site']) ? $invocation['site'] : $default_site;
714 // NULL is a synonym to '@self', although the latter is preferred.
715 if (!isset($site_record)) {
716 $site_record = '@self';
718 // If the first parameter is not a site alias record,
719 // then presume it is an alias name, and try to look up
721 if (!is_array($site_record)) {
722 $site_record = drush_sitealias_get_record($site_record);
724 $command = isset($invocation['command']) ? $invocation['command'] : $default_command;
725 $args = isset($invocation['args']) ? $invocation['args'] : [];
726 $command_options = isset($invocation['options']) ? $invocation['options'] : [];
727 $backend_options = isset($invocation['backend-options']) ? $invocation['backend-options'] : [];
728 // If $backend_options is passed in as a bool, interpret that as the value for 'integrate'
729 if (!is_array($common_backend_options)) {
730 $integrate = (bool)$common_backend_options;
731 $common_backend_options = ['integrate' => $integrate];
734 $command_options += $common_options;
735 $backend_options += $common_backend_options;
737 $backend_options = _drush_backend_adjust_options($site_record, $command, $command_options, $backend_options);
738 $backend_options += [
739 'drush-script' => NULL,
742 // Insure that contexts such as DRUSH_SIMULATE and NO_COLOR are included.
743 $command_options += _drush_backend_get_global_contexts($site_record);
745 // Add in command-specific options as well
746 // $command_options += drush_command_get_command_specific_options($site_record, $command);
748 $is_remote = array_key_exists('remote-host', $site_record);
750 // Add in preflight option contexts (--include et. al)
751 $preflightContextOptions = \Drush\Drush::config()->get(PreflightArgs::DRUSH_RUNTIME_CONTEXT_NAMESPACE, []);
752 $preflightContextOptions['local'] = \Drush\Drush::config()->get('runtime.local', false);
753 // If the command is local, also include the paths context.
755 $preflightContextOptions += \Drush\Drush::config()->get(PreflightArgs::DRUSH_CONFIG_PATH_NAMESPACE, []);
757 foreach ($preflightContextOptions as $key => $value) {
759 $command_options[$key] = $value;
763 // If the caller has requested it, don't pull the options from the alias
764 // into the command line, but use the alias name for dispatching.
765 if (!empty($backend_options['dispatch-using-alias']) && isset($site_record['#name'])) {
766 list($post_options, $commandline_options, $drush_global_options) = _drush_backend_classify_options([], $command_options, $backend_options);
767 $site_record_to_dispatch = '@' . ltrim($site_record['#name'], '@');
770 list($post_options, $commandline_options, $drush_global_options) = _drush_backend_classify_options($site_record, $command_options, $backend_options);
771 $site_record_to_dispatch = '';
773 if (array_key_exists('backend-simulate', $backend_options)) {
774 $drush_global_options['simulate'] = TRUE;
776 $site_record += ['path-aliases' => [], '#env-vars' => []];
777 $site_record['path-aliases'] += [
778 '%drush-script' => $backend_options['drush-script'],
781 $site = (array_key_exists('#name', $site_record) && !array_key_exists($site_record['#name'], $invocation_options)) ? $site_record['#name'] : $index++;
782 $invocation_options[$site] = [
783 'site-record' => $site_record,
784 'site-record-to-dispatch' => $site_record_to_dispatch,
785 'command' => $command,
787 'post-options' => $post_options,
788 'drush-global-options' => $drush_global_options,
789 'commandline-options' => $commandline_options,
790 'command-options' => $command_options,
791 'backend-options' => $backend_options,
795 // Calculate the length of the longest output label
796 $max_name_length = 0;
797 $label_separator = '';
798 if (!array_key_exists('no-label', $common_options) && (count($invocation_options) > 1)) {
799 $label_separator = array_key_exists('label-separator', $common_options) ? $common_options['label-separator'] : ' >> ';
800 foreach ($invocation_options as $site => $item) {
801 $backend_options = $item['backend-options'];
802 if (!array_key_exists('#output-label', $backend_options)) {
803 if (is_numeric($site)) {
804 $backend_options['#output-label'] = ' * [@self.' . $site;
805 $label_separator = '] ';
808 $backend_options['#output-label'] = $site;
810 $invocation_options[$site]['backend-options']['#output-label'] = $backend_options['#output-label'];
812 $name_len = strlen($backend_options['#output-label']);
813 if ($name_len > $max_name_length) {
814 $max_name_length = $name_len;
816 if (array_key_exists('#label-separator', $backend_options)) {
817 $label_separator = $backend_options['#label-separator'];
821 // Now pad out the output labels and add the label separator.
822 $reserve_margin = $max_name_length + strlen($label_separator);
823 foreach ($invocation_options as $site => $item) {
824 $backend_options = $item['backend-options'] + ['#output-label' => ''];
825 $invocation_options[$site]['backend-options']['#output-label'] = str_pad($backend_options['#output-label'], $max_name_length, " ") . $label_separator;
826 if ($reserve_margin) {
827 $invocation_options[$site]['drush-global-options']['reserve-margin'] = $reserve_margin;
831 // Now take our prepared options and generate the command strings
833 foreach ($invocation_options as $site => $item) {
834 $site_record = $item['site-record'];
835 $site_record_to_dispatch = $item['site-record-to-dispatch'];
836 $command = $item['command'];
837 $args = $item['args'];
838 $post_options = $item['post-options'];
839 $commandline_options = $item['commandline-options'];
840 $command_options = $item['command-options'];
841 $drush_global_options = $item['drush-global-options'];
842 $backend_options = $item['backend-options'];
843 $is_remote = array_key_exists('remote-host', $site_record);
846 (isset($site_record['root']) && ($site_record['root'] != drush_get_context('DRUSH_DRUPAL_ROOT'))) ||
847 (isset($site_record['uri']) && ($site_record['uri'] != drush_get_context('DRUSH_SELECTED_URI')));
848 $os = drush_os($site_record);
849 // If the caller did not pass in a specific path to drush, then we will
850 // use a default value. For commands that are being executed on the same
851 // machine, we will use DRUSH_COMMAND, which is the path to the drush.php
852 // that is running right now.
853 $drush_path = $site_record['path-aliases']['%drush-script'];
854 if (!$drush_path && !$is_remote && $is_different_site) {
855 $drush_path = DRUSH_COMMAND;
857 $env_vars = $site_record['#env-vars'];
858 $php = array_key_exists('php', $site_record) ? $site_record['php'] : (array_key_exists('php', $command_options) ? $command_options['php'] : NULL);
859 $drush_command_path = drush_build_drush_command($drush_path, $php, $os, $is_remote, $env_vars);
860 $cmd = _drush_backend_generate_command($site_record, $drush_command_path . " " . _drush_backend_argument_string($drush_global_options, $os) . " " . $site_record_to_dispatch . " " . $command, $args, $commandline_options, $backend_options) . ' 2>&1';
863 'post-options' => $post_options,
864 'backend-options' => $backend_options,
868 return _drush_backend_invoke($cmds, $common_backend_options, $context);
872 * Find all of the drush contexts that are used to cache global values and
873 * return them in an associative array.
875 function _drush_backend_get_global_contexts($site_record) {
877 $global_option_list = drush_get_global_options(FALSE);
878 foreach ($global_option_list as $global_key => $global_metadata) {
879 if (is_array($global_metadata)) {
881 if (!array_key_exists('never-propagate', $global_metadata)) {
882 if ((array_key_exists('propagate', $global_metadata))) {
883 $value = drush_get_option($global_key);
885 elseif ((array_key_exists('propagate-cli-value', $global_metadata))) {
886 $value = drush_get_option($global_key, '', 'cli');
888 elseif ((array_key_exists('context', $global_metadata))) {
889 // If the context is declared to be a 'local-context-only',
890 // then only put it in if this is a local dispatch.
891 if (!array_key_exists('local-context-only', $global_metadata) || !array_key_exists('remote-host', $site_record)) {
892 $value = drush_get_context($global_metadata['context'], []);
895 if (!empty($value) || ($value === '0')) {
896 $result[$global_key] = $value;
905 * Take all of the values in the $command_options array, and place each of
906 * them into one of the following result arrays:
908 * - $post_options: options to be encoded as JSON and written to the
909 * standard input of the drush subprocess being executed.
910 * - $commandline_options: options to be placed on the command line of the drush
912 * - $drush_global_options: the drush global options also go on the command
913 * line, but appear before the drush command name rather than after it.
915 * Also, this function may modify $backend_options.
917 function _drush_backend_classify_options($site_record, $command_options, &$backend_options) {
918 // In 'POST' mode (the default, remove everything (except the items marked 'never-post'
919 // in the global option list) from the commandline options and put them into the post options.
920 // The post options will be json-encoded and sent to the command via stdin
921 $global_option_list = drush_get_global_options(FALSE); // These should be in the command line.
922 $additional_global_options = [];
923 if (array_key_exists('additional-global-options', $backend_options)) {
924 $additional_global_options = $backend_options['additional-global-options'];
925 $command_options += $additional_global_options;
927 $method_post = ((!array_key_exists('method', $backend_options)) || ($backend_options['method'] == 'POST'));
929 $commandline_options = [];
930 $drush_global_options = [];
931 $drush_local_options = [];
932 $additional_backend_options = [];
933 foreach ($site_record as $key => $value) {
934 if (!in_array($key, drush_sitealias_site_selection_keys())) {
935 if ($key[0] == '#') {
936 $backend_options[$key] = $value;
938 if (!isset($command_options[$key])) {
939 if (array_key_exists($key, $global_option_list)) {
940 $command_options[$key] = $value;
945 if (array_key_exists('drush-local-options', $backend_options)) {
946 $drush_local_options = $backend_options['drush-local-options'];
947 $command_options += $drush_local_options;
949 if (!empty($backend_options['backend']) && empty($backend_options['interactive']) && empty($backend_options['fork'])) {
950 $drush_global_options['backend'] = '2';
952 foreach ($command_options as $key => $value) {
953 $global = array_key_exists($key, $global_option_list);
957 $propagate = (!array_key_exists('never-propagate', $global_option_list[$key]));
958 $special = (array_key_exists('never-post', $global_option_list[$key]));
960 // We will allow 'merge-pathlist' contexts to be propogated. Right now
961 // these are all 'local-context-only' options; if we allowed them to
962 // propogate remotely, then we would need to get the right path separator
963 // for the remote machine.
964 if (is_array($value) && array_key_exists('merge-pathlist', $global_option_list[$key])) {
965 $value = implode(PATH_SEPARATOR, $value);
969 // Just remove options that are designated as non-propagating
970 if ($propagate === TRUE) {
971 // In METHOD POST, move command options to post options
972 if ($method_post && ($special === FALSE)) {
973 $post_options[$key] = $value;
975 // In METHOD GET, ignore options with array values
976 elseif (!is_array($value)) {
977 if ($global || array_key_exists($key, $additional_global_options)) {
978 $drush_global_options[$key] = $value;
981 $commandline_options[$key] = $value;
986 return [$post_options, $commandline_options, $drush_global_options, $additional_backend_options];
990 * Create a new pipe with proc_open, and attempt to parse the output.
992 * We use proc_open instead of exec or others because proc_open is best
993 * for doing bi-directional pipes, and we need to pass data over STDIN
994 * to the remote script.
996 * Exec also seems to exhibit some strangeness in keeping the returned
997 * data intact, in that it modifies the newline characters.
1000 * The complete command line call to use.
1001 * @param post_options
1002 * An associative array to json-encode and pass to the remote script on stdin.
1003 * @param backend_options
1004 * Options for the invocation.
1007 * If no commands were executed, FALSE.
1009 * If one command was executed, this will return an associative array containing
1010 * the data from drush_backend_output(). The result code is stored
1011 * in $result['error_status'] (0 == no error).
1013 * If multiple commands were executed, this will return an associative array
1014 * containing one item, 'concurrent', which will contain a list of the different
1015 * backend invoke results from each concurrent command.
1017 function _drush_backend_invoke($cmds, $common_backend_options = [], $context = NULL) {
1018 if (\Drush\Drush::simulate() && !array_key_exists('override-simulated', $common_backend_options) && !array_key_exists('backend-simulate', $common_backend_options)) {
1019 foreach ($cmds as $cmd) {
1020 drush_print(dt('Simulating backend invoke: !cmd', ['!cmd' => $cmd['cmd']]));
1024 foreach ($cmds as $cmd) {
1025 drush_log(dt('Backend invoke: !cmd', ['!cmd' => $cmd['cmd']]), 'command');
1027 if (!empty($common_backend_options['interactive']) || !empty($common_backend_options['fork'])) {
1028 foreach ($cmds as $cmd) {
1029 $exec_cmd = $cmd['cmd'];
1030 if (array_key_exists('fork', $common_backend_options)) {
1031 $exec_cmd .= ' --quiet &';
1034 $result_code = drush_shell_proc_open($exec_cmd);
1035 $ret = ['error_status' => $result_code];
1039 $process_limit = drush_get_option_override($common_backend_options, 'concurrency', 1);
1040 $procs = _drush_backend_proc_open($cmds, $process_limit, $context);
1041 $procs = is_array($procs) ? $procs : [$procs];
1044 foreach ($procs as $site => $proc) {
1045 if (($proc['code'] == DRUSH_APPLICATION_ERROR) && isset($common_backend_options['integrate'])) {
1046 drush_set_error('DRUSH_APPLICATION_ERROR', dt("The external command could not be executed due to an application error."));
1049 if ($proc['output']) {
1050 $values = drush_backend_parse_output($proc['output'], $proc['backend-options'], $proc['outputted']);
1051 if (is_array($values)) {
1052 $values['site'] = $site;
1056 elseif (!array_key_exists('concurrent', $ret)) {
1057 $ret = ['concurrent' => [$ret, $values]];
1060 $ret['concurrent'][] = $values;
1064 $ret = drush_set_error('DRUSH_FRAMEWORK_ERROR', dt("The command could not be executed successfully (returned: !return, code: !code)", ["!return" => $proc['output'], "!code" => $proc['code']]));
1069 return empty($ret) ? FALSE : $ret;
1073 * Helper function that generates an anonymous site alias specification for
1074 * the given parameters.
1076 function drush_backend_generate_sitealias($backend_options) {
1077 // Ensure default values.
1078 $backend_options += [
1079 'remote-host' => NULL,
1080 'remote-user' => NULL,
1081 'ssh-options' => NULL,
1082 'drush-script' => NULL,
1086 'remote-host' => $backend_options['remote-host'],
1087 'remote-user' => $backend_options['remote-user'],
1088 'ssh-options' => $backend_options['ssh-options'],
1089 '#env-vars' => $backend_options['env-vars'],
1091 '%drush-script' => $backend_options['drush-script'],
1097 * Generate a command to execute.
1099 * @param site_record
1100 * An array containing information used to generate the command.
1102 * Optional. A remote host to execute the drush command on.
1104 * Optional. Defaults to the current user. If you specify this, you can choose which module to send.
1106 * Optional. Defaults to "-o PasswordAuthentication=no"
1108 * Optional. An associative array of environmental variables to prefix the Drush command with.
1110 * Optional; contains paths to folders and executables useful to the command.
1112 * Optional. Defaults to the current drush.php file on the local machine, and
1113 * to simply 'drush' (the drush script in the current PATH) on remote servers.
1114 * You may also specify a different drush.php script explicitly. You will need
1115 * to set this when calling drush on a remote server if 'drush' is not in the
1116 * PATH on that machine.
1118 * A defined drush command such as 'cron', 'status' or any of the available ones such as 'drush pm'.
1120 * An array of arguments for the command.
1121 * @param command_options
1122 * Optional. An array containing options to pass to the remote script.
1123 * Array items with a numeric key are treated as optional arguments to the
1124 * command. This parameter is a reference, as any options that have been
1125 * represented as either an option, or an argument will be removed. This
1126 * allows you to pass the left over options as a JSON encoded string,
1127 * without duplicating data.
1128 * @param backend_options
1129 * Optional. An array of options for the invocation.
1130 * @see drush_backend_invoke for documentation.
1133 * A text string representing a fully escaped command.
1135 function _drush_backend_generate_command($site_record, $command, $args = [], $command_options = [], $backend_options = []) {
1137 'remote-host' => NULL,
1138 'remote-user' => NULL,
1139 'ssh-options' => NULL,
1140 'path-aliases' => [],
1142 $backend_options += [
1146 $hostname = $site_record['remote-host'];
1147 $username = $site_record['remote-user'];
1148 $ssh_options = $site_record['ssh-options']; // TODO: update this (maybe make $site_record an AliasRecord)
1149 $os = drush_os($site_record);
1151 if (drush_is_local_host($hostname)) {
1155 foreach ($command_options as $key => $arg) {
1156 if (is_numeric($key)) {
1158 unset($command_options[$key]);
1163 foreach ($args as $arg) {
1164 $cmd[] = drush_escapeshellarg($arg, $os);
1166 $option_str = _drush_backend_argument_string($command_options, $os);
1167 if (!empty($option_str)) {
1168 $cmd[] = " " . $option_str;
1170 $command = implode(' ', array_filter($cmd, 'strlen'));
1171 if (isset($hostname)) {
1172 $username = (isset($username)) ? drush_escapeshellarg($username) . "@" : '';
1173 $ssh_options = $site_record['ssh-options']; // TODO: update
1174 $ssh_options = (isset($ssh_options)) ? $ssh_options : \Drush\Drush::config()->get('ssh.options', "-o PasswordAuthentication=no");
1177 $ssh_cmd[] = $ssh_options;
1178 if ($backend_options['#tty']) {
1181 $ssh_cmd[] = $username . drush_escapeshellarg($hostname);
1182 $ssh_cmd[] = drush_escapeshellarg($command . ' 2>&1');
1184 // Remove NULLs and separate with spaces
1185 $command = implode(' ', array_filter($ssh_cmd, 'strlen'));
1192 * Map the options to a string containing all the possible arguments and options.
1195 * Optional. An array containing options to pass to the remote script.
1196 * Array items with a numeric key are treated as optional arguments to the command.
1197 * This parameter is a reference, as any options that have been represented as either an option, or an argument will be removed.
1198 * This allows you to pass the left over options as a JSON encoded string, without duplicating data.
1200 * Optional. Defaults to 'GET'.
1201 * If this parameter is set to 'POST', the $data array will be passed to the script being called as a JSON encoded string over
1202 * the STDIN pipe of that process. This is preferable if you have to pass sensitive data such as passwords and the like.
1203 * For any other value, the $data array will be collapsed down into a set of command line options to the script.
1205 * A properly formatted and escaped set of arguments and options to append to the drush.php shell command.
1207 function _drush_backend_argument_string($data, $os = NULL) {
1210 foreach ($data as $key => $value) {
1211 if (!is_array($value) && !is_object($value) && isset($value)) {
1212 if (substr($key,0,1) != '#') {
1213 $options[$key] = $value;
1219 foreach ($options as $key => $value) {
1220 $option_str .= _drush_escape_option($key, $value, $os);
1227 * Return a properly formatted and escaped command line option
1230 * The name of the option.
1232 * The value of the option.
1235 * If the value is set to TRUE, this function will return " --key"
1236 * In other cases it will return " --key='value'"
1238 function _drush_escape_option($key, $value = TRUE, $os = NULL) {
1239 if ($value !== TRUE) {
1240 $option_str = " --$key=" . drush_escapeshellarg($value, $os);
1243 $option_str = " --$key";
1249 * Read options fron STDIN during POST requests.
1251 * This function will read any text from the STDIN pipe,
1252 * and attempts to generate an associative array if valid
1253 * JSON was received.
1256 * An associative array of options, if successfull. Otherwise FALSE.
1258 function _drush_backend_get_stdin() {
1259 $fp = fopen('php://stdin', 'r');
1260 // Windows workaround: we cannot count on stream_get_contents to
1261 // return if STDIN is reading from the keyboard. We will therefore
1262 // check to see if there are already characters waiting on the
1263 // stream (as there always should be, if this is a backend call),
1264 // and if there are not, then we will exit.
1265 // This code prevents drush from hanging forever when called with
1266 // --backend from the commandline; however, overall it is still
1267 // a futile effort, as it does not seem that backend invoke can
1268 // successfully write data to that this function can read,
1269 // so the argument list and command always come out empty. :(
1270 // Perhaps stream_get_contents is the problem, and we should use
1271 // the technique described here:
1272 // http://bugs.php.net/bug.php?id=30154
1273 // n.b. the code in that issue passes '0' for the timeout in stream_select
1274 // in a loop, which is not recommended.
1275 // Note that the following DOES work:
1276 // drush ev 'print(json_encode(array("test" => "XYZZY")));' | drush status --backend
1277 // So, redirecting input is okay, it is just the proc_open that is a problem.
1278 if (drush_is_windows()) {
1279 // Note that stream_select uses reference parameters, so we need variables (can't pass a constant NULL)
1283 // Question: might we need to wait a bit for STDIN to be ready,
1284 // even if the process that called us immediately writes our parameters?
1285 // Passing '100' for the timeout here causes us to hang indefinitely
1286 // when called from the shell.
1287 $changed_streams = stream_select($read, $write, $except, 0);
1288 // Return on error (FALSE) or no changed streams (0).
1289 // Oh, according to http://php.net/manual/en/function.stream-select.php,
1290 // stream_select will return FALSE for streams returned by proc_open.
1291 // That is not applicable to us, is it? Our stream is connected to a stream
1292 // created by proc_open, but is not a stream returned by proc_open.
1293 if ($changed_streams < 1) {
1297 stream_set_blocking($fp, FALSE);
1298 $string = stream_get_contents($fp);
1300 if (trim($string)) {
1301 return json_decode($string, TRUE);