PHP Internals — How foreach Works中文EN

foreach is a very commonly used control statement in PHP for looping over arrays.
Because it is so convenient and easy to use, it naturally hides a rather complicated implementation behind the scenes (transparent to the user).
Today, let's analyze together how foreach actually implements traversal of arrays (and objects).
This section involves a fair amount of compiler theory (lex and yacc), so if you find it hard to follow, you may want to read up on the related material first.

As we know, PHP is a scripting language, which means that the PHP code written by the user is ultimately interpreted and executed by the PHP interpreter. In particular, for PHP, all user-written PHP code is translated into virtual instructions (OPCODES) of PHP's virtual machine, the ZE, and then executed (see: PHP Internals — Opcodes).

Without going into the details, it means that any PHP script we write is ultimately translated into a series of instructions, and according to those instructions, the corresponding functions written in C are executed.

So what does foreach get translated into?

foreach($arr as $key => $val){
	echo $key . '=>' . $val . "n";
}

During the lexical analysis phase, foreach is recognized as a TOKEN: T_FOREACH.
During the syntax analysis phase, it is matched by the rule:

  unticked_statement:  //statement not bound to ticks
	//omitted
    |   T_FOREACH '(' variable T_AS
        { zend_do_foreach_begin(&$1, &$2, &$3, &$4, 1 TSRMLS_CC); }
        foreach_variable foreach_optional_arg ')' { zend_do_foreach_cont(&$1, &$2, &$4, &$6, &$7 TSRMLS_CC); }
        foreach_statement { zend_do_foreach_end(&$1, &$4 TSRMLS_CC); }
    |   T_FOREACH '(' expr_without_variable T_AS
        { zend_do_foreach_begin(&$1, &$2, &$3, &$4, 0 TSRMLS_CC); }
        variable foreach_optional_arg ')' { zend_check_writable_variable(&$6); zend_do_foreach_cont(&$1, &$2, &$4, &$6, &$7 TSRMLS_CC); }
        foreach_statement { zend_do_foreach_end(&$1, &$4 TSRMLS_CC); }
	//omitted
;

Analyzing this grammar rule carefully, we can see that for:
foreach($arr as $key => $val){
echo $key . '=>' . $val ."n";
}
it will be parsed as:

	T_FOREACH '(' variable T_AS   { zend_do_foreach_begin('foreach', '(', $arr, 'as', 1 TSRMLS_CC); }
    foreach_variable  foreach_optional_arg(T_DOUBLE_ARROW  foreach_variable)   ')'  { zend_do_foreach_cont('foreach', '(', 'as', $key, $val TSRMLS_CC); }
    foreach_satement {zend_do_foreach_end('foreach', 'as');}

Next, let's look at foreach_statement:
it is in fact just a code block that expresses our echo $key . '=>' . $val ."n";
T_ECHO expr;
Obviously, the core of implementing foreach is the following 3 functions:
zend_do_foreach_begin
zend_do_foreach_cont
zend_do_foreach_end
Among them, zend_do_foreach_begin (the code is too long, so I'll just write pseudocode) mainly does:
1. Records the current opline line number (recorded for later jumps)
2. RESETs the array (points the internal pointer to the first element)
3. Gets a temporary variable ($val)
4. Sets the OPCODE FE_FETCH for fetching the variable, storing the result in the temporary variable from step 3
4. Records the line number of the OPCODES that fetch the variable
As for zend_do_foreach_cont:
1. Determines whether it is a reference based on the u.EA.type of foreach_variable
2. Adjusts the FE_FETCH mode generated in zend_do_foreach_begin according to whether it is a reference
3. Initializes the loop based on the line number of the variable-fetching OPCODES recorded in zend_do_foreach_begin (mainly handling loops inside loops: do_begin_loop)
Finally, zend_do_foreach_end:
1. Sets the ZEND_JMP OPCODES based on the line number information recorded in zend_do_foreach_begin
2. Based on the current line number, sets the next opline after the loop body, so as to break out of the loop
3. Ends the loop (handling loops inside loops: do_end_loop)
4. Cleans up temporary variables
Of course, between zend_do_foreach_cont and zend_do_foreach_end, the statement code of foreach_satement is filled in during the syntax analysis phase.

And so the OPCODES line for foreach is implemented.
For example, for the sample code at the beginning, the finally generated OPCODES are:

filename:       /home/huixinchen/foreach.php
function name:  (null)
number of ops:  17
compiled vars:  !0 = $arr, !1 = $key, !2 = $val
line     #  op                           fetch          ext  return  operands
-------------------------------------------------------------------------------
   2     0  SEND_VAL                                                 1
         1  SEND_VAL                                                 100
         2  DO_FCALL                                      2          'range'
         3  ASSIGN                                                   !0, $0
   3     4  FE_RESET                                         $2      !0, ->14
         5  FE_FETCH                                         $3      $2, ->14
         6  ZEND_OP_DATA                                     ~5
         7  ASSIGN                                                   !2, $3
         8  ASSIGN                                                   !1, ~5
   4     9  CONCAT                                           ~7      !1, '-'
        10  CONCAT                                           ~8      ~7, !2
        11  CONCAT                                           ~9      ~8, '%0A'
        12  ECHO                                                     ~9
   5    13  JMP                                                      ->5
        14  SWITCH_FREE                                              $2
   7    15  RETURN                                                   1
        16* ZEND_HANDLE_EXCEPTION

We notice that the operand of op2 of FE_FETCH is 14, which is the opline after JMP. That is, after the last array element has been fetched, when FE_FETCH fails, it jumps to opline line 14, thus ending the loop.
And the operand of op1 of opline line 15 points to FE_FETCH, that is, an unconditional jump to opline line 5, thus implementing the loop.

Appendix:

void zend_do_foreach_begin(znode *foreach_token, znode *open_brackets_token, znode *array, znode *as_token, int variable TSRMLS_DC)
{
    zend_op *opline;
    zend_bool is_variable;
    zend_bool push_container = 0;
    zend_op dummy_opline;
    if (variable) {
		//is it an anonymous array
        if (zend_is_function_or_method_call(array)) {
			//is it a function return value
            is_variable = 0;
        } else {
            is_variable = 1;
        }
        /* use the brackets to record the opline number of FE_RESET */
        open_brackets_token->u.opline_num = get_next_op_number(CG(active_op_array));
        zend_do_end_variable_parse(BP_VAR_W, 0 TSRMLS_CC); //fetch the array/object, corresponds to zend_do_begin_variable_parse
        if (CG(active_op_array)->last > 0 &&
            CG(active_op_array)->opcodes[CG(active_op_array)->last-1].opcode == ZEND_FETCH_OBJ_W) {
            /* Only lock the container if we are fetching from a real container and not $this */
            if (CG(active_op_array)->opcodes[CG(active_op_array)->last-1].op1.op_type == IS_VAR) {
                CG(active_op_array)->opcodes[CG(active_op_array)->last-1].extended_value |= ZEND_FETCH_ADD_LOCK;
                push_container = 1;
            }
        }
    } else {
        is_variable = 0;
        open_brackets_token->u.opline_num = get_next_op_number(CG(active_op_array));
    }
    foreach_token->u.opline_num = get_next_op_number(CG(active_op_array)); //record the array Reset Opline number
    opline = get_next_op(CG(active_op_array) TSRMLS_CC); //generate the Reset array Opcode
    opline->opcode = ZEND_FE_RESET;
    opline->result.op_type = IS_VAR;
    opline->result.u.var = get_temporary_variable(CG(active_op_array));
    opline->op1 = *array;
    SET_UNUSED(opline->op2);
    opline->extended_value = is_variable ? ZEND_FE_RESET_VARIABLE : 0;
    dummy_opline.result = opline->result;
    if (push_container) {
        dummy_opline.op1 = CG(active_op_array)->opcodes[CG(active_op_array)->last-2].op1;
    } else {
        znode tmp;
        tmp.op_type = IS_UNUSED;
        dummy_opline.op1 = tmp;
    }
    zend_stack_push(&CG(foreach_copy_stack), (void *) &dummy_opline, sizeof(zend_op));
    as_token->u.opline_num = get_next_op_number(CG(active_op_array)); //record the loop start point
    opline = get_next_op(CG(active_op_array) TSRMLS_CC);
    opline->opcode = ZEND_FE_FETCH;
    opline->result.op_type = IS_VAR;
    opline->result.u.var = get_temporary_variable(CG(active_op_array));
    opline->op1 = dummy_opline.result;    //the array being operated on
    opline->extended_value = 0;
    SET_UNUSED(opline->op2);
    opline = get_next_op(CG(active_op_array) TSRMLS_CC);
    opline->opcode = ZEND_OP_DATA; //the attached operand when key is used, ignored when foreach contains no key
    SET_UNUSED(opline->op1);
    SET_UNUSED(opline->op2);
    SET_UNUSED(opline->result);
}

void zend_do_foreach_cont(znode *foreach_token, const znode *open_brackets_token, const znode *as_token, znode *value, znode *key TSRMLS_DC)
{
    zend_op *opline;
    znode dummy, value_node;
    zend_bool assign_by_ref=0;
    opline = &CG(active_op_array)->opcodes[as_token->u.opline_num]; //get the FE_FETCH Opline
    if (key->op_type != IS_UNUSED) {
        znode *tmp;//swap key and val
        tmp = key;
        key = value;
        value = tmp;
        opline->extended_value |= ZEND_FE_FETCH_WITH_KEY; //indicates that both key and val need to be fetched
    }
    if ((key->op_type != IS_UNUSED) && (key->u.EA.type & ZEND_PARSED_REFERENCE_VARIABLE)) {
		//key cannot be fetched by reference
        zend_error(E_COMPILE_ERROR, "Key element cannot be a reference");
    }
    if (value->u.EA.type & ZEND_PARSED_REFERENCE_VARIABLE) {
		//fetch the value by reference
        assign_by_ref = 1;
        if (!(opline-1)->extended_value) {
			//based on the previous Opline of FE_FETCH, i.e. the extended value of the array fetch, determine whether the array is an anonymous array
            zend_error(E_COMPILE_ERROR, "Cannot create references to elements of a temporary array expression");
        }
        opline->extended_value |= ZEND_FE_FETCH_BYREF; //indicates fetching by reference
        CG(active_op_array)->opcodes[foreach_token->u.opline_num].extended_value |= ZEND_FE_RESET_REFERENCE; //reset the original array
    } else {
        zend_op *foreach_copy;
        zend_op *fetch = &CG(active_op_array)->opcodes[foreach_token->u.opline_num];
        zend_op *end = &CG(active_op_array)->opcodes[open_brackets_token->u.opline_num];
        /* Change "write context" into "read context" */
        fetch->extended_value = 0;  /* reset ZEND_FE_RESET_VARIABLE */
        while (fetch != end) {
            --fetch;
            if (fetch->opcode == ZEND_FETCH_DIM_W && fetch->op2.op_type == IS_UNUSED) {
                zend_error(E_COMPILE_ERROR, "Cannot use [] for reading");
            }
            fetch->opcode -= 3; /* FETCH_W -> FETCH_R */
        }
        /* prevent double SWITCH_FREE */
        zend_stack_top(&CG(foreach_copy_stack), (void **) &foreach_copy);
        foreach_copy->op1.op_type = IS_UNUSED;
    }
    value_node = opline->result;
    if (assign_by_ref) {
        zend_do_end_variable_parse(value, BP_VAR_W, 0 TSRMLS_CC); //fetch the value (by reference)
        zend_do_assign_ref(NULL, value, &value_node TSRMLS_CC);//indicates that the type of value node is IS_VAR
    } else {
        zend_do_assign(&dummy, value, &value_node TSRMLS_CC); //fetch the copy value
        zend_do_free(&dummy TSRMLS_CC);
    }
    if (key->op_type != IS_UNUSED) {
        znode key_node;
        opline = &CG(active_op_array)->opcodes[as_token->u.opline_num+1];
        opline->result.op_type = IS_TMP_VAR;
        opline->result.u.EA.type = 0;
        opline->result.u.opline_num = get_temporary_variable(CG(active_op_array));
        key_node = opline->result;
        zend_do_assign(&dummy, key, &key_node TSRMLS_CC);
        zend_do_free(&dummy TSRMLS_CC);
    }
    do_begin_loop(TSRMLS_C);
    INC_BPC(CG(active_op_array));
}

void zend_do_foreach_end(znode *foreach_token, znode *as_token TSRMLS_DC)
{
    zend_op *container_ptr;
    zend_op *opline = get_next_op(CG(active_op_array) TSRMLS_CC); //generate the JMP opcode
    opline->opcode = ZEND_JMP;
    opline->op1.u.opline_num = as_token->u.opline_num; //set JMP to the FE_FETCH opline line
    SET_UNUSED(opline->op1);
    SET_UNUSED(opline->op2);
    CG(active_op_array)->opcodes[foreach_token->u.opline_num].op2.u.opline_num = get_next_op_number(CG(active_op_array)); //set the opline line to break out of the loop
    CG(active_op_array)->opcodes[as_token->u.opline_num].op2.u.opline_num = get_next_op_number(CG(active_op_array)); //same as above
    do_end_loop(as_token->u.opline_num, 1 TSRMLS_CC); //set up for nested loops
    zend_stack_top(&CG(foreach_copy_stack), (void **) &container_ptr);
    generate_free_foreach_copy(container_ptr TSRMLS_CC);
    zend_stack_del_top(&CG(foreach_copy_stack));
    DEC_BPC(CG(active_op_array)); //set up for PHP interactive mode
}