7 * When a drush command is called with the --backend option,
8 * it will buffer all output, and instead return a JSON encoded
9 * string containing all relevant information on the command that
12 * Through this mechanism, it is possible for Drush commands to
15 * There are many cases where a command might wish to call another
16 * command in its own process, to allow the calling command to
17 * intercept and act on any errors that may occur in the script that
20 * A simple example is if there exists an 'update' command for running
21 * update.php on a specific site. The original command might download
22 * a newer version of a module for installation on a site, and then
23 * run the update script in a separate process, so that in the case
24 * of an error running a hook_update_n function, the module can revert
25 * to a previously made database backup, and the previously installed code.
27 * By calling the script in a separate process, the calling script is insulated
28 * from any error that occurs in the called script, to the level that if a
29 * php code error occurs (ie: misformed file, missing parenthesis, whatever),
30 * it is still able to reliably handle any problems that occur.
32 * This is nearly a RESTful API. @see http://en.wikipedia.org/wiki/REST
35 * http://[server]/[apipath]/[command]?[arg1]=[value1],[arg2]=[value2]
38 * [apipath] [command] --[arg1]=[value1] --[arg2]=[value2] --backend
40 * [apipath] in this case will be the path to the drush.php file.
41 * [command] is the command you would call, for instance 'status'.
43 * GET parameters will be passed as options to the script.
44 * POST parameters will be passed to the script as a JSON encoded associative array over STDIN.
46 * Because of this standard interface, Drush commands can also be executed on
47 * external servers through SSH pipes, simply by prepending, 'ssh username@server.com'
48 * in front of the command.
50 * If the key-based ssh authentication has been set up between the servers,
51 * this will just work. By default, drush is configured to disallow password
52 * authentication; if you would like to enter a password for every connection,
53 * then in your drushrc.php file, set $options['ssh-options'] so that it does NOT
54 * include '-o PasswordAuthentication=no'. See examples/example.drushrc.php.
56 * The results from backend API calls can be fetched via a call to
57 * drush_backend_get_result().
60 use Drush\Log\LogLevel;
61 use Drush\Preflight\PreflightArgs;
64 * Identify the JSON encoded output from a command.
66 * Note that Drush now outputs a null ("\0") before the DRUSH_BACKEND_OUTPUT_DELIMITER,
67 * but this null occurs where this constant is output rather than being
68 * included in the define. This is done to maintain compatibility with
69 * older versions of Drush, so that Drush-7.x can correctly parse backend messages
70 * from calls made to Drush-5.x and earlier. The null is removed via trim().
72 define('DRUSH_BACKEND_OUTPUT_START', 'DRUSH_BACKEND_OUTPUT_START>>>');
73 define('DRUSH_BACKEND_OUTPUT_DELIMITER', DRUSH_BACKEND_OUTPUT_START . '%s<<<DRUSH_BACKEND_OUTPUT_END');
76 * Identify JSON encoded "packets" embedded inside of backend
77 * output; used to send out-of-band information durring a backend
78 * invoke call (currently only used for log and error messages).
80 define('DRUSH_BACKEND_PACKET_START', "DRUSH_BACKEND:");
81 define('DRUSH_BACKEND_PACKET_PATTERN', "\0" . DRUSH_BACKEND_PACKET_START . "%s\n\0");
84 * The backend result is the original PHP data structure (usually an array)
85 * used to generate the output for the current command.
87 function drush_backend_set_result($value) {
88 if (\Drush\Drush::backend()) {
89 drush_set_context('BACKEND_RESULT', $value);
94 * Retrieves the results from the last call to backend_invoke.
97 * An associative array containing information from the last
98 * backend invoke. The keys in the array include:
100 * - output: This item contains the textual output of
101 * the command that was executed.
102 * - object: Contains the PHP object representation of the
103 * result of the command.
104 * - self: The self object contains the alias record that was
105 * used to select the bootstrapped site when the command was
107 * - error_status: This item returns the error status for the
108 * command. Zero means "no error".
109 * - log: The log item contains an array of log messages from
110 * the command execution ordered chronologically. Each log
111 * entery is an associative array. A log entry contains
113 * o type: The type of log entry, such as 'notice' or 'warning'
114 * o message: The log message
115 * o timestamp: The time that the message was logged
116 * o memory: Available memory at the time that the message was logged
117 * o error: The error code associated with the log message
118 * (only for log entries whose type is 'error')
119 * - error_log: The error_log item contains another representation
120 * of entries from the log. Only log entries whose 'error' item
121 * is set will appear in the error log. The error log is an
122 * associative array whose key is the error code, and whose value
123 * is an array of messages--one message for every log entry with
124 * the same error code.
125 * - context: The context item contains a representation of all option
126 * values that affected the operation of the command, including both
127 * the command line options, options set in a drushrc.php configuration
128 * files, and options set from the alias record used with the command.
130 function drush_backend_get_result() {
131 return drush_get_context('BACKEND_RESULT');
135 * Print the json-encoded output of this command, including the
136 * encoded log records, context information, etc.
138 function drush_backend_output() {
141 // Strip out backend commands.
142 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), ["\0" => "\\0"]);
143 $packet_regex = str_replace("\n", "", $packet_regex);
144 $data['output'] = preg_replace("/$packet_regex/s", '', drush_backend_output_collect(NULL));
146 if (drush_get_context('DRUSH_QUIET', FALSE)) {
150 $result_object = drush_backend_get_result();
151 if (isset($result_object)) {
152 $data['object'] = $result_object;
155 $error = drush_get_error();
156 $data['error_status'] = ($error) ? $error : DRUSH_SUCCESS;
158 $data['log'] = drush_get_log(); // Append logging information
159 // The error log is a more specific version of the log, and may be used by calling
160 // scripts to check for specific errors that have occurred.
161 $data['error_log'] = drush_get_error_log();
162 // If there is a @self record, then include it in the result
163 $self_record = drush_sitealias_get_record('@self');
164 if (!empty($self_record)) {
165 $site_context = drush_get_context('site', []);
166 unset($site_context['config-file']);
167 unset($site_context['context-path']);
168 unset($self_record['loaded-config']);
169 unset($self_record['#name']);
170 $data['self'] = array_merge($site_context, $self_record);
173 // Return the options that were set at the end of the process.
174 $data['context'] = drush_get_merged_options();
175 printf("\0" . DRUSH_BACKEND_OUTPUT_DELIMITER, json_encode($data));
179 * Callback to collect backend command output.
181 function drush_backend_output_collect($string) {
183 if (!isset($string)) {
192 * Output buffer functions that discards all output but backend packets.
194 function drush_backend_output_discard($string) {
195 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), ["\0" => "\\0"]);
196 $packet_regex = str_replace("\n", "", $packet_regex);
197 if (preg_match_all("/$packet_regex/s", $string, $matches)) {
198 return implode('', $matches[0]);
203 * Output a backend packet if we're running as backend.
206 * The packet to send.
208 * Data for the command.
211 * A boolean indicating whether the command was output.
213 function drush_backend_packet($packet, $data) {
214 if (\Drush\Drush::backend()) {
215 $data['packet'] = $packet;
216 $data = json_encode($data);
217 // We use 'fwrite' instead of 'drush_print' here because
218 // this backend packet is out-of-band data.
219 fwrite(STDERR, sprintf(DRUSH_BACKEND_PACKET_PATTERN, $data));
227 * Parse output returned from a Drush command.
230 * The output of a drush command
232 * Integrate the errors and log messages from the command into the current process.
234 * Whether output has already been handled.
237 * An associative array containing the data from the external command, or the string parameter if it
238 * could not be parsed successfully.
240 function drush_backend_parse_output($string, $backend_options = [], $outputted = FALSE) {
241 $regex = sprintf(DRUSH_BACKEND_OUTPUT_DELIMITER, '(.*)');
243 preg_match("/$regex/s", $string, $match);
245 if (!empty($match) && $match[1]) {
246 // we have our JSON encoded string
248 // remove the match we just made and any non printing characters
249 $string = trim(str_replace(sprintf(DRUSH_BACKEND_OUTPUT_DELIMITER, $match[1]), '', $string));
252 if (!empty($output)) {
253 $data = json_decode($output, TRUE);
254 if (is_array($data)) {
255 _drush_backend_integrate($data, $backend_options, $outputted);
263 * Integrate log messages and error statuses into the current
266 * Output produced by the called script will be printed if we didn't print it
267 * on the fly, errors will be set, and log messages will be logged locally, if
268 * not already logged.
271 * The associative array returned from the external command.
273 * Whether output has already been handled.
275 function _drush_backend_integrate($data, $backend_options, $outputted) {
276 // In 'integrate' mode, logs and errors have already been handled
277 // by drush_backend_packet (sender) drush_backend_parse_packets (receiver - us)
278 // during incremental output. We therefore do not need to call drush_set_error
279 // or drush_log here. The exception is if the sender is an older version of
280 // Drush (version 4.x) that does not send backend packets, then we will
281 // not have processed the log entries yet, and must print them here.
282 $received_packets = drush_get_context('DRUSH_RECEIVED_BACKEND_PACKETS', FALSE);
283 if (is_array($data['log']) && $backend_options['log'] && (!$received_packets)) {
284 foreach($data['log'] as $log) {
285 $message = is_array($log['message']) ? implode("\n", $log['message']) : $log['message'];
286 if (isset($backend_options['#output-label'])) {
287 $message = $backend_options['#output-label'] . $message;
289 if (isset($log['error']) && $backend_options['integrate']) {
290 drush_set_error($log['error'], $message);
292 elseif ($backend_options['integrate']) {
293 drush_log($message, $log['type']);
297 // Output will either be printed, or buffered to the drush_backend_output command.
298 // If the output has already been printed, then we do not need to show it again on a failure.
300 if (drush_cmp_error('DRUSH_APPLICATION_ERROR') && !empty($data['output'])) {
301 drush_set_error("DRUSH_APPLICATION_ERROR", dt("Output from failed command :\n !output", ['!output' => $data['output']]));
303 elseif ($backend_options['output']) {
304 _drush_backend_print_output($data['output'], $backend_options);
310 * Supress log message output during backend integrate.
312 function _drush_backend_integrate_log($entry) {
316 * Call an external command using proc_open.
319 * An array of records containing the following elements:
320 * 'cmd' - The command to execute, already properly escaped
321 * 'post-options' - An associative array that will be JSON encoded
322 * and passed to the script being called. Objects are not allowed,
323 * as they do not json_decode gracefully.
324 * 'backend-options' - Options that control the operation of the backend invoke
326 * An array of commands to execute. These commands already need to be properly escaped.
327 * In this case, post-options will default to empty, and a default output label will
330 * An associative array that will be JSON encoded and passed to the script being called.
331 * Objects are not allowed, as they do not json_decode gracefully.
334 * False if the command could not be executed, or did not return any output.
335 * If it executed successfully, it returns an associative array containing the command
336 * called, the output of the command, and the error code of the command.
338 function _drush_backend_proc_open($cmds, $process_limit, $context = NULL) {
340 0 => ["pipe", "r"], // stdin is a pipe that the child will read from
341 1 => ["pipe", "w"], // stdout is a pipe that the child will write to
344 $open_processes = [];
346 $process_limit = max($process_limit, 1);
347 $is_windows = drush_is_windows();
348 // Loop through processes until they all close, having a nap as needed.
350 while (count($open_processes) || count($cmds)) {
352 if (count($cmds) && (count($open_processes) < $process_limit)) {
353 // Pop the site and command (key / value) from the cmds array
355 $cmd = current($cmds);
359 if (is_array($cmd)) {
361 $post_options = $cmd['post-options'];
362 $backend_options = $cmd['backend-options'];
367 $backend_options = [];
369 $backend_options += [
370 '#output-label' => '',
371 '#process-read-size' => 4096,
374 drush_log($backend_options['#output-label'] . $c);
375 $process['process'] = proc_open($c, $descriptorspec, $process['pipes'], null, null, ['context' => $context]);
376 if (is_resource($process['process'])) {
378 fwrite($process['pipes'][0], json_encode($post_options)); // pass the data array in a JSON encoded string
380 // If we do not close stdin here, then we cause a deadlock;
381 // see: http://drupal.org/node/766080#comment-4309936
382 // If we reimplement interactive commands to also use
383 // _drush_proc_open, then clearly we would need to keep
385 fclose($process['pipes'][0]);
387 $process['info'] = stream_get_meta_data($process['pipes'][1]);
388 stream_set_blocking($process['pipes'][1], FALSE);
389 stream_set_timeout($process['pipes'][1], 1);
390 $bucket[$site]['cmd'] = $c;
391 $bucket[$site]['output'] = '';
392 $bucket[$site]['remainder'] = '';
393 $bucket[$site]['backend-options'] = $backend_options;
394 $bucket[$site]['end_of_output'] = FALSE;
395 $bucket[$site]['outputted'] = FALSE;
396 $open_processes[$site] = $process;
398 // Reset the $nap_time variable as there might be output to process next
402 // Set up to call stream_select(). See:
403 // http://php.net/manual/en/function.stream-select.php
404 // We can't use stream_select on Windows, because it doesn't work for
405 // streams returned by proc_open.
410 $except_streams = [];
411 foreach ($open_processes as $site => &$current_process) {
412 if (isset($current_process['pipes'][1])) {
413 $read_streams[] = $current_process['pipes'][1];
416 // Wait up to 2s for data to become ready on one of the read streams.
417 if (count($read_streams)) {
418 $ss_result = stream_select($read_streams, $write_streams, $except_streams, 2);
419 // If stream_select returns a error, then fallback to using $nap_time.
420 if ($ss_result !== FALSE) {
426 foreach ($open_processes as $site => &$current_process) {
427 if (isset($current_process['pipes'][1])) {
428 // Collect output from stdout
429 $bucket[$site][1] = '';
430 $info = stream_get_meta_data($current_process['pipes'][1]);
432 if (!feof($current_process['pipes'][1]) && !$info['timed_out']) {
433 $string = $bucket[$site]['remainder'] . fread($current_process['pipes'][1], $backend_options['#process-read-size']);
434 $bucket[$site]['remainder'] = '';
435 $output_end_pos = strpos($string, DRUSH_BACKEND_OUTPUT_START);
436 if ($output_end_pos !== FALSE) {
437 $trailing_string = substr($string, 0, $output_end_pos);
438 $trailing_remainder = '';
439 // If there is any data in the trailing string (characters prior
440 // to the backend output start), then process any backend packets
442 if (strlen($trailing_string) > 0) {
443 drush_backend_parse_packets($trailing_string, $trailing_remainder, $bucket[$site]['backend-options']);
445 // If there is any data remaining in the trailing string after
446 // the backend packets are removed, then print it.
447 if (strlen($trailing_string) > 0) {
448 _drush_backend_print_output($trailing_string . $trailing_remainder, $bucket[$site]['backend-options']);
449 $bucket[$site]['outputted'] = TRUE;
451 $bucket[$site]['end_of_output'] = TRUE;
453 if (!$bucket[$site]['end_of_output']) {
454 drush_backend_parse_packets($string, $bucket[$site]['remainder'], $bucket[$site]['backend-options']);
455 // Pass output through.
456 _drush_backend_print_output($string, $bucket[$site]['backend-options']);
457 if (strlen($string) > 0) {
458 $bucket[$site]['outputted'] = TRUE;
461 $bucket[$site][1] .= $string;
462 $bucket[$site]['output'] .= $string;
463 $info = stream_get_meta_data($current_process['pipes'][1]);
466 // Reset the $nap_time variable as there might be output to process
468 if (strlen($string) > 0) {
473 fclose($current_process['pipes'][1]);
474 unset($current_process['pipes'][1]);
475 // close the pipe , set a marker
477 // Reset the $nap_time variable as there might be output to process
483 // if both pipes are closed for the process, remove it from active loop and add a new process to open.
484 $bucket[$site]['code'] = proc_close($current_process['process']);
485 unset($open_processes[$site]);
487 // Reset the $nap_time variable as there might be output to process next
493 // We should sleep for a bit if we need to, up to a maximum of 1/10 of a
496 usleep(max($nap_time * 500, 100000));
500 // TODO: Handle bad proc handles
508 * Print the output received from a call to backend invoke,
509 * adding the label to the head of each line if necessary.
511 function _drush_backend_print_output($output_string, $backend_options) {
512 if ($backend_options['output'] && !empty($output_string)) {
513 $output_label = array_key_exists('#output-label', $backend_options) ? $backend_options['#output-label'] : FALSE;
514 if (!empty($output_label)) {
515 // Remove one, and only one newline from the end of the
516 // string. Else we'll get an extra 'empty' line.
517 foreach (explode("\n", preg_replace('/\\n$/', '', $output_string)) as $line) {
518 fwrite(STDOUT, $output_label . rtrim($line) . "\n");
522 fwrite(STDOUT, $output_string);
528 * Parse out and remove backend packet from the supplied string and
529 * invoke the commands.
531 function drush_backend_parse_packets(&$string, &$remainder, $backend_options) {
533 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), ["\0" => "\\0"]);
534 $packet_regex = str_replace("\n", "", $packet_regex);
535 if (preg_match_all("/$packet_regex/s", $string, $match, PREG_PATTERN_ORDER)) {
536 drush_set_context('DRUSH_RECEIVED_BACKEND_PACKETS', TRUE);
537 foreach ($match[1] as $packet_data) {
538 $entry = (array) json_decode($packet_data);
539 if (is_array($entry) && isset($entry['packet'])) {
540 $function = 'drush_backend_packet_' . $entry['packet'];
541 if (function_exists($function)) {
542 $function($entry, $backend_options);
545 drush_log(dt("Unknown backend packet @packet", ['@packet' => $entry['packet']]), LogLevel::INFO);
549 drush_log(dt("Malformed backend packet"), LogLevel::ERROR);
550 drush_log(dt("Bad packet: @packet", ['@packet' => print_r($entry, TRUE)]), LogLevel::DEBUG);
551 drush_log(dt("String is: @str", ['@str' => $packet_data]), LogLevel::DEBUG);
554 $string = preg_replace("/$packet_regex/s", '', $string);
556 // Check to see if there is potentially a partial packet remaining.
557 // We only care about the last null; if there are any nulls prior
558 // to the last one, they would have been removed above if they were
559 // valid drush packets.
560 $embedded_null = strrpos($string, "\0");
561 if ($embedded_null !== FALSE) {
562 // We will consider everything after $embedded_null to be part of
563 // the $remainder string if:
564 // - the embedded null is less than strlen(DRUSH_BACKEND_OUTPUT_START)
565 // from the end of $string (that is, there might be a truncated
566 // backend packet header, or the truncated backend output start
569 // - the embedded null is followed by DRUSH_BACKEND_PACKET_START
570 // (that is, the terminating null for that packet has not been
571 // read into our buffer yet)
572 if (($embedded_null + strlen(DRUSH_BACKEND_OUTPUT_START) >= strlen($string)) || (substr($string, $embedded_null + 1, strlen(DRUSH_BACKEND_PACKET_START)) == DRUSH_BACKEND_PACKET_START)) {
573 $remainder = substr($string, $embedded_null);
574 $string = substr($string, 0, $embedded_null);
580 * Backend command for setting errors.
582 function drush_backend_packet_set_error($data, $backend_options) {
583 if (!$backend_options['integrate']) {
587 if (array_key_exists('#output-label', $backend_options)) {
588 $output_label = $backend_options['#output-label'];
590 drush_set_error($data['error'], $data['message'], $output_label);
594 * Default options for backend_invoke commands.
596 function _drush_backend_adjust_options($site_record, $command, $command_options, $backend_options) {
597 // By default, if the caller does not specify a value for 'output', but does
598 // specify 'integrate' === FALSE, then we will set output to FALSE. Otherwise we
599 // will allow it to default to TRUE.
600 if ((array_key_exists('integrate', $backend_options)) && ($backend_options['integrate'] === FALSE) && (!array_key_exists('output', $backend_options))) {
601 $backend_options['output'] = FALSE;
603 $has_site_specification = array_key_exists('root', $site_record) || array_key_exists('uri', $site_record);
604 $result = $backend_options + [
610 'dispatch-using-alias' => !$has_site_specification,
612 // Convert '#integrate' et. al. into backend options
613 foreach ($command_options as $key => $value) {
614 if (substr($key,0,1) === '#') {
615 $result[substr($key,1)] = $value;
622 * Execute a new local or remote command in a new process.
624 * n.b. Prefer drush_invoke_process() to this function.
627 * An array of command records to execute. Each record should contain:
629 * An array containing information used to generate the command.
631 * Optional. A remote host to execute the drush command on.
633 * Optional. Defaults to the current user. If you specify this, you can choose which module to send.
635 * Optional. Defaults to "-o PasswordAuthentication=no"
637 * Optional. An associative array of environmental variables to prefix the Drush command with.
639 * Optional; contains paths to folders and executables useful to the command.
641 * Optional. Defaults to the current drush.php file on the local machine, and
642 * to simply 'drush' (the drush script in the current PATH) on remote servers.
643 * You may also specify a different drush.php script explicitly. You will need
644 * to set this when calling drush on a remote server if 'drush' is not in the
645 * PATH on that machine.
647 * A defined drush command such as 'cron', 'status' or any of the available ones such as 'drush pm'.
649 * An array of arguments for the command.
651 * Optional. An array containing options to pass to the remote script.
652 * Array items with a numeric key are treated as optional arguments to the
655 * Optional. Additional parameters that control the operation of the invoke.
657 * Optional. Defaults to 'GET'.
658 * If this parameter is set to 'POST', the $data array will be passed
659 * to the script being called as a JSON encoded string over the STDIN
660 * pipe of that process. This is preferable if you have to pass
661 * sensitive data such as passwords and the like.
662 * For any other value, the $data array will be collapsed down into a
663 * set of command line options to the script.
665 * Optional. Defaults to TRUE.
666 * If TRUE, any error statuses will be integrated into the current
667 * process. This might not be what you want, if you are writing a
668 * command that operates on multiple sites.
670 * Optional. Defaults to TRUE.
671 * If TRUE, any log messages will be integrated into the current
674 * Optional. Defaults to TRUE.
675 * If TRUE, output from the command will be synchronously printed to
678 * Optional. Defaults to the current drush.php file on the local
679 * machine, and to simply 'drush' (the drush script in the current
680 * PATH) on remote servers. You may also specify a different drush.php
681 * script explicitly. You will need to set this when calling drush on
682 * a remote server if 'drush' is not in the PATH on that machine.
683 * 'dispatch-using-alias'
684 * Optional. Defaults to FALSE.
685 * If specified as a non-empty value the drush command will be
686 * dispatched using the alias name on the command line, instead of
687 * the options from the alias being added to the command line
689 * @param common_options
690 * Optional. Merged in with the options for each invocation.
691 * @param backend_options
692 * Optional. Merged in with the backend options for each invocation.
693 * @param default_command
694 * Optional. Used as the 'command' for any invocation that does not
695 * define a command explicitly.
696 * @param default_site
697 * Optional. Used as the 'site' for any invocation that does not
698 * define a site explicitly.
700 * Optional. Passed in to proc_open if provided.
703 * If the command could not be completed successfully, FALSE.
704 * If the command was completed, this will return an associative array containing the data from drush_backend_output().
706 function drush_backend_invoke_concurrent($invocations, $common_options = [], $common_backend_options = [], $default_command = NULL, $default_site = NULL, $context = NULL) {
709 // Slice and dice our options in preparation to build a command string
710 $invocation_options = [];
711 foreach ($invocations as $invocation) {
712 $site_record = isset($invocation['site']) ? $invocation['site'] : $default_site;
713 // NULL is a synonym to '@self', although the latter is preferred.
714 if (!isset($site_record)) {
715 $site_record = '@self';
717 // If the first parameter is not a site alias record,
718 // then presume it is an alias name, and try to look up
720 if (!is_array($site_record)) {
721 $site_record = drush_sitealias_get_record($site_record);
723 $command = isset($invocation['command']) ? $invocation['command'] : $default_command;
724 $args = isset($invocation['args']) ? $invocation['args'] : [];
725 $command_options = isset($invocation['options']) ? $invocation['options'] : [];
726 $backend_options = isset($invocation['backend-options']) ? $invocation['backend-options'] : [];
727 // If $backend_options is passed in as a bool, interpret that as the value for 'integrate'
728 if (!is_array($common_backend_options)) {
729 $integrate = (bool)$common_backend_options;
730 $common_backend_options = ['integrate' => $integrate];
733 $command_options += $common_options;
734 $backend_options += $common_backend_options;
736 $backend_options = _drush_backend_adjust_options($site_record, $command, $command_options, $backend_options);
737 $backend_options += [
738 'drush-script' => NULL,
741 // Insure that contexts such as DRUSH_SIMULATE and NO_COLOR are included.
742 $command_options += _drush_backend_get_global_contexts($site_record);
744 // Add in command-specific options as well
745 // $command_options += drush_command_get_command_specific_options($site_record, $command);
747 // Add in preflight option contexts (--include et. al)
748 $preflightContextOptions =
749 \Drush\Drush::config()->get(PreflightArgs::DRUSH_RUNTIME_CONTEXT_NAMESPACE, []) +
750 \Drush\Drush::config()->get(PreflightArgs::DRUSH_CONFIG_PATH_NAMESPACE, []);
751 $preflightContextOptions['local'] = \Drush\Drush::config()->get('runtime.local', false);
752 foreach ($preflightContextOptions as $key => $value) {
754 $command_options[$key] = $value;
758 // If the caller has requested it, don't pull the options from the alias
759 // into the command line, but use the alias name for dispatching.
760 if (!empty($backend_options['dispatch-using-alias']) && isset($site_record['#name'])) {
761 list($post_options, $commandline_options, $drush_global_options) = _drush_backend_classify_options([], $command_options, $backend_options);
762 $site_record_to_dispatch = '@' . ltrim($site_record['#name'], '@');
765 list($post_options, $commandline_options, $drush_global_options) = _drush_backend_classify_options($site_record, $command_options, $backend_options);
766 $site_record_to_dispatch = '';
768 if (array_key_exists('backend-simulate', $backend_options)) {
769 $drush_global_options['simulate'] = TRUE;
771 $site_record += ['path-aliases' => [], '#env-vars' => []];
772 $site_record['path-aliases'] += [
773 '%drush-script' => $backend_options['drush-script'],
776 $site = (array_key_exists('#name', $site_record) && !array_key_exists($site_record['#name'], $invocation_options)) ? $site_record['#name'] : $index++;
777 $invocation_options[$site] = [
778 'site-record' => $site_record,
779 'site-record-to-dispatch' => $site_record_to_dispatch,
780 'command' => $command,
782 'post-options' => $post_options,
783 'drush-global-options' => $drush_global_options,
784 'commandline-options' => $commandline_options,
785 'command-options' => $command_options,
786 'backend-options' => $backend_options,
790 // Calculate the length of the longest output label
791 $max_name_length = 0;
792 $label_separator = '';
793 if (!array_key_exists('no-label', $common_options) && (count($invocation_options) > 1)) {
794 $label_separator = array_key_exists('label-separator', $common_options) ? $common_options['label-separator'] : ' >> ';
795 foreach ($invocation_options as $site => $item) {
796 $backend_options = $item['backend-options'];
797 if (!array_key_exists('#output-label', $backend_options)) {
798 if (is_numeric($site)) {
799 $backend_options['#output-label'] = ' * [@self.' . $site;
800 $label_separator = '] ';
803 $backend_options['#output-label'] = $site;
805 $invocation_options[$site]['backend-options']['#output-label'] = $backend_options['#output-label'];
807 $name_len = strlen($backend_options['#output-label']);
808 if ($name_len > $max_name_length) {
809 $max_name_length = $name_len;
811 if (array_key_exists('#label-separator', $backend_options)) {
812 $label_separator = $backend_options['#label-separator'];
816 // Now pad out the output labels and add the label separator.
817 $reserve_margin = $max_name_length + strlen($label_separator);
818 foreach ($invocation_options as $site => $item) {
819 $backend_options = $item['backend-options'] + ['#output-label' => ''];
820 $invocation_options[$site]['backend-options']['#output-label'] = str_pad($backend_options['#output-label'], $max_name_length, " ") . $label_separator;
821 if ($reserve_margin) {
822 $invocation_options[$site]['drush-global-options']['reserve-margin'] = $reserve_margin;
826 // Now take our prepared options and generate the command strings
828 foreach ($invocation_options as $site => $item) {
829 $site_record = $item['site-record'];
830 $site_record_to_dispatch = $item['site-record-to-dispatch'];
831 $command = $item['command'];
832 $args = $item['args'];
833 $post_options = $item['post-options'];
834 $commandline_options = $item['commandline-options'];
835 $command_options = $item['command-options'];
836 $drush_global_options = $item['drush-global-options'];
837 $backend_options = $item['backend-options'];
838 $is_remote = array_key_exists('remote-host', $site_record);
841 (isset($site_record['root']) && ($site_record['root'] != drush_get_context('DRUSH_DRUPAL_ROOT'))) ||
842 (isset($site_record['uri']) && ($site_record['uri'] != drush_get_context('DRUSH_SELECTED_URI')));
843 $os = drush_os($site_record);
844 // If the caller did not pass in a specific path to drush, then we will
845 // use a default value. For commands that are being executed on the same
846 // machine, we will use DRUSH_COMMAND, which is the path to the drush.php
847 // that is running right now.
848 $drush_path = $site_record['path-aliases']['%drush-script'];
849 if (!$drush_path && !$is_remote && $is_different_site) {
850 $drush_path = DRUSH_COMMAND;
852 $env_vars = $site_record['#env-vars'];
853 $php = array_key_exists('php', $site_record) ? $site_record['php'] : (array_key_exists('php', $command_options) ? $command_options['php'] : NULL);
854 $drush_command_path = drush_build_drush_command($drush_path, $php, $os, $is_remote, $env_vars);
855 $cmd = _drush_backend_generate_command($site_record, $drush_command_path . " " . _drush_backend_argument_string($drush_global_options, $os) . " " . $site_record_to_dispatch . " " . $command, $args, $commandline_options, $backend_options) . ' 2>&1';
858 'post-options' => $post_options,
859 'backend-options' => $backend_options,
863 return _drush_backend_invoke($cmds, $common_backend_options, $context);
867 * Find all of the drush contexts that are used to cache global values and
868 * return them in an associative array.
870 function _drush_backend_get_global_contexts($site_record) {
872 $global_option_list = drush_get_global_options(FALSE);
873 foreach ($global_option_list as $global_key => $global_metadata) {
874 if (is_array($global_metadata)) {
876 if (!array_key_exists('never-propagate', $global_metadata)) {
877 if ((array_key_exists('propagate', $global_metadata))) {
878 $value = drush_get_option($global_key);
880 elseif ((array_key_exists('propagate-cli-value', $global_metadata))) {
881 $value = drush_get_option($global_key, '', 'cli');
883 elseif ((array_key_exists('context', $global_metadata))) {
884 // If the context is declared to be a 'local-context-only',
885 // then only put it in if this is a local dispatch.
886 if (!array_key_exists('local-context-only', $global_metadata) || !array_key_exists('remote-host', $site_record)) {
887 $value = drush_get_context($global_metadata['context'], []);
890 if (!empty($value) || ($value === '0')) {
891 $result[$global_key] = $value;
900 * Take all of the values in the $command_options array, and place each of
901 * them into one of the following result arrays:
903 * - $post_options: options to be encoded as JSON and written to the
904 * standard input of the drush subprocess being executed.
905 * - $commandline_options: options to be placed on the command line of the drush
907 * - $drush_global_options: the drush global options also go on the command
908 * line, but appear before the drush command name rather than after it.
910 * Also, this function may modify $backend_options.
912 function _drush_backend_classify_options($site_record, $command_options, &$backend_options) {
913 // In 'POST' mode (the default, remove everything (except the items marked 'never-post'
914 // in the global option list) from the commandline options and put them into the post options.
915 // The post options will be json-encoded and sent to the command via stdin
916 $global_option_list = drush_get_global_options(FALSE); // These should be in the command line.
917 $additional_global_options = [];
918 if (array_key_exists('additional-global-options', $backend_options)) {
919 $additional_global_options = $backend_options['additional-global-options'];
920 $command_options += $additional_global_options;
922 $method_post = ((!array_key_exists('method', $backend_options)) || ($backend_options['method'] == 'POST'));
924 $commandline_options = [];
925 $drush_global_options = [];
926 $drush_local_options = [];
927 $additional_backend_options = [];
928 foreach ($site_record as $key => $value) {
929 if (!in_array($key, drush_sitealias_site_selection_keys())) {
930 if ($key[0] == '#') {
931 $backend_options[$key] = $value;
933 if (!isset($command_options[$key])) {
934 if (array_key_exists($key, $global_option_list)) {
935 $command_options[$key] = $value;
940 if (array_key_exists('drush-local-options', $backend_options)) {
941 $drush_local_options = $backend_options['drush-local-options'];
942 $command_options += $drush_local_options;
944 if (!empty($backend_options['backend']) && empty($backend_options['interactive']) && empty($backend_options['fork'])) {
945 $drush_global_options['backend'] = '2';
947 foreach ($command_options as $key => $value) {
948 $global = array_key_exists($key, $global_option_list);
952 $propagate = (!array_key_exists('never-propagate', $global_option_list[$key]));
953 $special = (array_key_exists('never-post', $global_option_list[$key]));
955 // We will allow 'merge-pathlist' contexts to be propogated. Right now
956 // these are all 'local-context-only' options; if we allowed them to
957 // propogate remotely, then we would need to get the right path separator
958 // for the remote machine.
959 if (is_array($value) && array_key_exists('merge-pathlist', $global_option_list[$key])) {
960 $value = implode(PATH_SEPARATOR, $value);
964 // Just remove options that are designated as non-propagating
965 if ($propagate === TRUE) {
966 // In METHOD POST, move command options to post options
967 if ($method_post && ($special === FALSE)) {
968 $post_options[$key] = $value;
970 // In METHOD GET, ignore options with array values
971 elseif (!is_array($value)) {
972 if ($global || array_key_exists($key, $additional_global_options)) {
973 $drush_global_options[$key] = $value;
976 $commandline_options[$key] = $value;
981 return [$post_options, $commandline_options, $drush_global_options, $additional_backend_options];
985 * Create a new pipe with proc_open, and attempt to parse the output.
987 * We use proc_open instead of exec or others because proc_open is best
988 * for doing bi-directional pipes, and we need to pass data over STDIN
989 * to the remote script.
991 * Exec also seems to exhibit some strangeness in keeping the returned
992 * data intact, in that it modifies the newline characters.
995 * The complete command line call to use.
996 * @param post_options
997 * An associative array to json-encode and pass to the remote script on stdin.
998 * @param backend_options
999 * Options for the invocation.
1002 * If no commands were executed, FALSE.
1004 * If one command was executed, this will return an associative array containing
1005 * the data from drush_backend_output(). The result code is stored
1006 * in $result['error_status'] (0 == no error).
1008 * If multiple commands were executed, this will return an associative array
1009 * containing one item, 'concurrent', which will contain a list of the different
1010 * backend invoke results from each concurrent command.
1012 function _drush_backend_invoke($cmds, $common_backend_options = [], $context = NULL) {
1013 if (\Drush\Drush::simulate() && !array_key_exists('override-simulated', $common_backend_options) && !array_key_exists('backend-simulate', $common_backend_options)) {
1014 foreach ($cmds as $cmd) {
1015 drush_print(dt('Simulating backend invoke: !cmd', ['!cmd' => $cmd['cmd']]));
1019 foreach ($cmds as $cmd) {
1020 drush_log(dt('Backend invoke: !cmd', ['!cmd' => $cmd['cmd']]), 'command');
1022 if (!empty($common_backend_options['interactive']) || !empty($common_backend_options['fork'])) {
1023 foreach ($cmds as $cmd) {
1024 $exec_cmd = $cmd['cmd'];
1025 if (array_key_exists('fork', $common_backend_options)) {
1026 $exec_cmd .= ' --quiet &';
1029 $result_code = drush_shell_proc_open($exec_cmd);
1030 $ret = ['error_status' => $result_code];
1034 $process_limit = drush_get_option_override($common_backend_options, 'concurrency', 1);
1035 $procs = _drush_backend_proc_open($cmds, $process_limit, $context);
1036 $procs = is_array($procs) ? $procs : [$procs];
1039 foreach ($procs as $site => $proc) {
1040 if (($proc['code'] == DRUSH_APPLICATION_ERROR) && isset($common_backend_options['integrate'])) {
1041 drush_set_error('DRUSH_APPLICATION_ERROR', dt("The external command could not be executed due to an application error."));
1044 if ($proc['output']) {
1045 $values = drush_backend_parse_output($proc['output'], $proc['backend-options'], $proc['outputted']);
1046 if (is_array($values)) {
1047 $values['site'] = $site;
1051 elseif (!array_key_exists('concurrent', $ret)) {
1052 $ret = ['concurrent' => [$ret, $values]];
1055 $ret['concurrent'][] = $values;
1059 $ret = drush_set_error('DRUSH_FRAMEWORK_ERROR', dt("The command could not be executed successfully (returned: !return, code: !code)", ["!return" => $proc['output'], "!code" => $proc['code']]));
1064 return empty($ret) ? FALSE : $ret;
1068 * Helper function that generates an anonymous site alias specification for
1069 * the given parameters.
1071 function drush_backend_generate_sitealias($backend_options) {
1072 // Ensure default values.
1073 $backend_options += [
1074 'remote-host' => NULL,
1075 'remote-user' => NULL,
1076 'ssh-options' => NULL,
1077 'drush-script' => NULL,
1081 'remote-host' => $backend_options['remote-host'],
1082 'remote-user' => $backend_options['remote-user'],
1083 'ssh-options' => $backend_options['ssh-options'],
1084 '#env-vars' => $backend_options['env-vars'],
1086 '%drush-script' => $backend_options['drush-script'],
1092 * Generate a command to execute.
1094 * @param site_record
1095 * An array containing information used to generate the command.
1097 * Optional. A remote host to execute the drush command on.
1099 * Optional. Defaults to the current user. If you specify this, you can choose which module to send.
1101 * Optional. Defaults to "-o PasswordAuthentication=no"
1103 * Optional. An associative array of environmental variables to prefix the Drush command with.
1105 * Optional; contains paths to folders and executables useful to the command.
1107 * Optional. Defaults to the current drush.php file on the local machine, and
1108 * to simply 'drush' (the drush script in the current PATH) on remote servers.
1109 * You may also specify a different drush.php script explicitly. You will need
1110 * to set this when calling drush on a remote server if 'drush' is not in the
1111 * PATH on that machine.
1113 * A defined drush command such as 'cron', 'status' or any of the available ones such as 'drush pm'.
1115 * An array of arguments for the command.
1116 * @param command_options
1117 * Optional. An array containing options to pass to the remote script.
1118 * Array items with a numeric key are treated as optional arguments to the
1119 * command. This parameter is a reference, as any options that have been
1120 * represented as either an option, or an argument will be removed. This
1121 * allows you to pass the left over options as a JSON encoded string,
1122 * without duplicating data.
1123 * @param backend_options
1124 * Optional. An array of options for the invocation.
1125 * @see drush_backend_invoke for documentation.
1128 * A text string representing a fully escaped command.
1130 function _drush_backend_generate_command($site_record, $command, $args = [], $command_options = [], $backend_options = []) {
1132 'remote-host' => NULL,
1133 'remote-user' => NULL,
1134 'ssh-options' => NULL,
1135 'path-aliases' => [],
1137 $backend_options += [
1141 $hostname = $site_record['remote-host'];
1142 $username = $site_record['remote-user'];
1143 $ssh_options = $site_record['ssh-options']; // TODO: update this (maybe make $site_record an AliasRecord)
1144 $os = drush_os($site_record);
1146 if (drush_is_local_host($hostname)) {
1150 foreach ($command_options as $key => $arg) {
1151 if (is_numeric($key)) {
1153 unset($command_options[$key]);
1158 foreach ($args as $arg) {
1159 $cmd[] = drush_escapeshellarg($arg, $os);
1161 $option_str = _drush_backend_argument_string($command_options, $os);
1162 if (!empty($option_str)) {
1163 $cmd[] = " " . $option_str;
1165 $command = implode(' ', array_filter($cmd, 'strlen'));
1166 if (isset($hostname)) {
1167 $username = (isset($username)) ? drush_escapeshellarg($username) . "@" : '';
1168 $ssh_options = $site_record['ssh-options']; // TODO: update
1169 $ssh_options = (isset($ssh_options)) ? $ssh_options : \Drush\Drush::config()->get('ssh.options', "-o PasswordAuthentication=no");
1172 $ssh_cmd[] = $ssh_options;
1173 if ($backend_options['#tty']) {
1176 $ssh_cmd[] = $username . drush_escapeshellarg($hostname);
1177 $ssh_cmd[] = drush_escapeshellarg($command . ' 2>&1');
1179 // Remove NULLs and separate with spaces
1180 $command = implode(' ', array_filter($ssh_cmd, 'strlen'));
1187 * Map the options to a string containing all the possible arguments and options.
1190 * Optional. An array containing options to pass to the remote script.
1191 * Array items with a numeric key are treated as optional arguments to the command.
1192 * This parameter is a reference, as any options that have been represented as either an option, or an argument will be removed.
1193 * This allows you to pass the left over options as a JSON encoded string, without duplicating data.
1195 * Optional. Defaults to 'GET'.
1196 * If this parameter is set to 'POST', the $data array will be passed to the script being called as a JSON encoded string over
1197 * the STDIN pipe of that process. This is preferable if you have to pass sensitive data such as passwords and the like.
1198 * For any other value, the $data array will be collapsed down into a set of command line options to the script.
1200 * A properly formatted and escaped set of arguments and options to append to the drush.php shell command.
1202 function _drush_backend_argument_string($data, $os = NULL) {
1205 foreach ($data as $key => $value) {
1206 if (!is_array($value) && !is_object($value) && isset($value)) {
1207 if (substr($key,0,1) != '#') {
1208 $options[$key] = $value;
1214 foreach ($options as $key => $value) {
1215 $option_str .= _drush_escape_option($key, $value, $os);
1222 * Return a properly formatted and escaped command line option
1225 * The name of the option.
1227 * The value of the option.
1230 * If the value is set to TRUE, this function will return " --key"
1231 * In other cases it will return " --key='value'"
1233 function _drush_escape_option($key, $value = TRUE, $os = NULL) {
1234 if ($value !== TRUE) {
1235 $option_str = " --$key=" . drush_escapeshellarg($value, $os);
1238 $option_str = " --$key";
1244 * Read options fron STDIN during POST requests.
1246 * This function will read any text from the STDIN pipe,
1247 * and attempts to generate an associative array if valid
1248 * JSON was received.
1251 * An associative array of options, if successfull. Otherwise FALSE.
1253 function _drush_backend_get_stdin() {
1254 $fp = fopen('php://stdin', 'r');
1255 // Windows workaround: we cannot count on stream_get_contents to
1256 // return if STDIN is reading from the keyboard. We will therefore
1257 // check to see if there are already characters waiting on the
1258 // stream (as there always should be, if this is a backend call),
1259 // and if there are not, then we will exit.
1260 // This code prevents drush from hanging forever when called with
1261 // --backend from the commandline; however, overall it is still
1262 // a futile effort, as it does not seem that backend invoke can
1263 // successfully write data to that this function can read,
1264 // so the argument list and command always come out empty. :(
1265 // Perhaps stream_get_contents is the problem, and we should use
1266 // the technique described here:
1267 // http://bugs.php.net/bug.php?id=30154
1268 // n.b. the code in that issue passes '0' for the timeout in stream_select
1269 // in a loop, which is not recommended.
1270 // Note that the following DOES work:
1271 // drush ev 'print(json_encode(array("test" => "XYZZY")));' | drush status --backend
1272 // So, redirecting input is okay, it is just the proc_open that is a problem.
1273 if (drush_is_windows()) {
1274 // Note that stream_select uses reference parameters, so we need variables (can't pass a constant NULL)
1278 // Question: might we need to wait a bit for STDIN to be ready,
1279 // even if the process that called us immediately writes our parameters?
1280 // Passing '100' for the timeout here causes us to hang indefinitely
1281 // when called from the shell.
1282 $changed_streams = stream_select($read, $write, $except, 0);
1283 // Return on error (FALSE) or no changed streams (0).
1284 // Oh, according to http://php.net/manual/en/function.stream-select.php,
1285 // stream_select will return FALSE for streams returned by proc_open.
1286 // That is not applicable to us, is it? Our stream is connected to a stream
1287 // created by proc_open, but is not a stream returned by proc_open.
1288 if ($changed_streams < 1) {
1292 stream_set_blocking($fp, FALSE);
1293 $string = stream_get_contents($fp);
1295 if (trim($string)) {
1296 return json_decode($string, TRUE);