Skip to content

Add support for "write-once" properties #5186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from

Conversation

kocsismate
Copy link
Member

@kocsismate kocsismate commented Feb 18, 2020

Currently, the implementation makes use of the final keyword, although the actual name is to be decided.

RFC: https://wiki.php.net/rfc/write_once_properties

@kocsismate kocsismate added the RFC label Feb 18, 2020
@HallofFamer
Copy link

I think this is a good idea, and the keyword final is quite appropriate here. Java has final class/method/property, while C# doesnt have the final keyword and instead use other keywords like 'sealed' and 'readonly'. From the aspect of consistency, the keyword final is very suitable for immutable properties too.

@kocsismate
Copy link
Member Author

@HallofFamer Unfortunately, I won't propose final as a keyword in the RFC :(

In PHP, final currently prevents a class or a method to be overridden. In the sense of this, using final for "immutable" properties would be a mistake.

@kocsismate kocsismate changed the title Add support for final properties Add support for "write-once" properties Feb 18, 2020
@nikic
Copy link
Member

nikic commented Feb 20, 2020

This doesn't implement the "references are not allowed" part yet, right?

@kocsismate
Copy link
Member Author

kocsismate commented Feb 20, 2020

I would say I only had a (really bad) try at it, but haven't looked at it again since then...

if (Z_TYPE_P(retval) != IS_UNDEF && Z_TYPE_P(retval) != IS_OBJECT && Z_TYPE_P(retval) != IS_RESOURCE) {

@mart-k
Copy link

mart-k commented Feb 20, 2020

Maybe I missed it in RFC, but what happens when you call next($this->c); ? Will it move the internal pointer (of an array) or throw an exception?

@kocsismate
Copy link
Member Author

@mart-k I've just done a quick test, and actually, next() throws an exception. :( I think it should work since it's needed for reading, so I'll add a test case for this and then try to fix the issue. Also I'll modify the RFC to mention the expected behaviour. Thank you for pointing this out!

@nikic
Copy link
Member

nikic commented Feb 20, 2020

@kocsismate Err, why would next() work? It modifies the array, so it should not be permitted.

Note that foreach(), ArrayIterator etc all work independently of the internal array pointer and can work fine with a readonly property.

@kocsismate
Copy link
Member Author

Note that foreach(), ArrayIterator etc all work independently of the internal array pointer and can work fine with a readonly property.

Oh, I thought that the internal pointer is used during iteration, so my reasoning was that then next() should be allowed. Sorry for this. But the testcase + another example come in handy indeed.

@mart-k
Copy link

mart-k commented Feb 20, 2020

One of the reasons why I pointed next() out is that there are whole family of similar functions that read/write array's "internal pointer". For example what happens if you call current()? It is read only and doesn't change the pointer, but if next(), each() etc. give error, will allowing current() even make sense?

@kocsismate
Copy link
Member Author

@mart-k Calling current() currently works. My opinion is that it doesn't make sense to disallow those functions that only read the pointer... But Nikita should be more informed.

@nikic
Copy link
Member

nikic commented Feb 21, 2020

@kocsismate Right, I don't see why these need any special consideration. They are normal by-value or by-reference functions, depending on whether they perform modification or not.

@someniatko
Copy link

someniatko commented Feb 25, 2020

What about clonable objects? There is a popular #withX() pattern which first clones an object, and then assigns the prop of cloned object new value, therefore preserving immutability. It is used e.g. in PSR-7 interfaces (methods like withHeader()).

Like this:

<?php

class C
{
    writeonce public int $prop;
    
    public function __construct(int $prop)
    {
        $this->prop = $prop;
    }

    public function withProp(int $prop): self
    {
         $that = clone $this;
         $that->prop = $prop; // will fail?

         return $that;
    }
}

$instance = new C(1);
$newInstance = $instance->withProp(2);

@kocsismate
Copy link
Member Author

@someniatko Cloning is currently under debate. My point of view is that it's not a prerequisite for writeonce/readonly properties since you can create a completely new instance instead of using clone. It's not that comfortable though, but I'd like to come up with a way to support mutability of immutable properties. :)

@AnrDaemon
Copy link

IMO, this suggestion is extremely unsettling. Probably on the level of singletons - it produces a sudden behavior change that can't be explained by surrounding code logic.

If you want special actions taken for writing data to the property, a more generic approach can be taken.

Something like

[static] property [typehint] $name [write methodName];

where methodName is a class method with signature identical to __set() except for matching typehints. F.e. function(string $name, ?typehint $value).
If no setter method declared, it will be public read-only property. If a setter is declared, an attempt to write to the property outside the object/class will be redirected to the setter.

It is arguable if a setter requires $name in its arguments, but I think this can increase usability of the feature by allowing for setter reuse without writing a lot of proxy methods.

@AaronBernabeu
Copy link

What would happen with property reflection? I haven't read about reflection in the RFC, I guess it should be taken into account and add a method to identify whether a property is immutable or not.

@kocsismate
Copy link
Member Author

@Aaronidas yeah, I'll definitely add a new ReflectionProperty::isReadonly() method. It's also worth a mention in the RFC. Thanks for the reminder.

@someniatko
Copy link

someniatko commented Feb 27, 2020

@kocsismate i think you should be consistent and name it isWriteonce(), please.

@kocsismate
Copy link
Member Author

@someniatko sorry, I was a bit unclear: the actual method name depends on the outcome of the secondary vote which is about the keyword choice.

@kocsismate kocsismate force-pushed the final-properties branch 4 times, most recently from 59f47ad to b0a042d Compare March 8, 2020 21:46
Copy link
Member

@nikic nikic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm missing tests for the "lazy initialization" pattern, that is unset the uninit property and use __get() to perform initialization.

I also saw that you do allow acquiring references via foreach-by-ref -- is that intentional? Basically, we should either be allowing all references (and preventing assignments to them) or forbidding all references. Allowing some references combines the disadvantages of both :)

@kocsismate
Copy link
Member Author

@nikic My newest commit addresses some of your concerns, like adding the missing tests and inheritance checks. Thanks to your help I think I managed to make the "foreach-by-ref"-related tests work.

However, lazy initialization didn't yet work because zend_verify_prop_assignable_by_ref() is called when calling getters and I implemented the "Cannot modify final properties by reference" check there. I am not sure if I did the right thing, but I disabled this validation in this very case, and now it's all good... I guess the magic_unset_success.phpt test doesn't work because of a similar reason (and that's why it says reference while in fact it isn't involved).

@@ -1502,6 +1532,14 @@ ZEND_API zval *zend_std_get_static_property_with_info(zend_class_entry *ce, zend
ret = CE_STATIC_MEMBERS(ce) + property_info->offset;
ZVAL_DEINDIRECT(ret);

if (UNEXPECTED((type == BP_VAR_RW || type == BP_VAR_W) && property_info->flags & ZEND_ACC_FINAL && Z_TYPE_P(ret) != IS_UNDEF)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... I think there's a pretty big problem here :(

We currently compile the operands of property accesses with the property access mode, i.e. Foo::$bar->baz = 1 is going to perform a W fetch on the static property, rather than the R fetch you would expect.

This means that code like this is going to error, while it should not:

<?php
class Foo {
    final public static stdClass $prop;
}

Foo::$prop = new stdClass;
Foo::$prop->bar = 1;

Probably more test coverage for modification of objects stored in readonly properties is needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to fix this in #3865, but ran into some compatibility issues with SimpleXML.

Copy link
Member Author

@kocsismate kocsismate Mar 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds really bad. :/ But I'll add more tests of course. Do you have any idea if this issue is a deal breaker or could be somehow circumvented? Btw is it worth to have SimpleXML in core if it has issues like this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, I opened #5250 and started a thread at https://externals.io/message/108950.

Btw is it worth to have SimpleXML in core if it has issues like this?

It's not, but people will complain if we drop it :)

if (result) ZVAL_ERROR(result);
return 0;
}
break;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this code actually do anything? I don't see any corresponding compiler code that sets these flags, so I would expect it to never run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, not much. I put it as an earlier experimentation (without checking the compiler), but forgot to remove it.

Copy link
Member Author

@kocsismate kocsismate Mar 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, or is it possible that I'll need this code? Currently, I'm trying to implement the restriction for the ZEND_FETCH_OBJ_UNSET opcode. I see that zend_fetch_property_address() is called from the opcode handler with flags=0. However, should I have this piece of code while setting flags to 1 then maybe unset($foo->a[0]) could be checked? Or is there a better way?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think you're going to need at least the UNSET portion of this, in order to distinguish between unset($foo->a[0]) and unset($foo->a->b) (otherwise you could just check for BP_VAR_UNSET). You'll have to add the corresponding flag in the compiler (like it is done for the two flags that are checked above). One value was freed up in PHP 8, so it should fit in.

@nikic
Copy link
Member

nikic commented Mar 11, 2020

I'm missing some tests for $obj->prop += 1 or similar. I think it might not be handled right now.

@@ -0,0 +1,104 @@
--TEST--
Test that final properties can't be mutated by assignment operators
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests might not be very useful because all of them boil down to simple assignments from the perspective of "write-once" properties

}

try {
$foo->property2 ??= "foo";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't managed to make this case work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only does an assignment if the property is null. So it's correct that this doesn't error as-is, you'd have to use a different default value.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a little bastard :'D But thanks for the info, it was too difficult for me to discover how it really works.

@nikic
Copy link
Member

nikic commented Mar 11, 2020

One more thing to test would be by-ref passing. For example sort($obj->prop).

@@ -1804,7 +1804,7 @@ ZEND_VM_HELPER(zend_fetch_static_prop_helper, ANY, ANY, int type)

SAVE_OPLINE();

if (UNEXPECTED(zend_fetch_static_property_address(&prop, NULL, opline->extended_value & ~ZEND_FETCH_OBJ_FLAGS, type, opline->extended_value & ZEND_FETCH_OBJ_FLAGS OPLINE_CC EXECUTE_DATA_CC) != SUCCESS)) {
if (UNEXPECTED(zend_fetch_static_property_address(&prop, NULL, opline->extended_value & ~(type == BP_VAR_UNSET ? ZEND_FETCH_DIM_UNSET_FLAG : ZEND_FETCH_OBJ_FLAGS), type, opline->extended_value & (type == BP_VAR_UNSET ? ZEND_FETCH_DIM_UNSET_FLAG : ZEND_FETCH_OBJ_FLAGS) OPLINE_CC EXECUTE_DATA_CC) != SUCCESS)) {
Copy link
Member Author

@kocsismate kocsismate Mar 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still a work in progress part

Copy link
Contributor

@TysonAndre TysonAndre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could add tests involving unserialize() and final properties - For example, setting final properties with or without defaults in __unserialize() or __wakeup(), or no magic methods at all, or after unserialization (should throw if defined, and not throw if undefined).
Unserializers have been problematic, e.g. for typed properties https://bugs.php.net/bug.php?id=79002 was changed in 7.4.2. I'd guess the implementation probably works, though

Also, are there tests of references to fields of properties (e.g. what is the intended behavior for preg_match('/abc/', 'abcdef', $x->prop['field'])) - I skimmed the tests and didn't see any

if (Z_TYPE_P(variable_ptr) != IS_UNDEF) {
if (UNEXPECTED(Z_PROP_FLAG_P(variable_ptr) != IS_PROP_UNINIT && prop_info && prop_info->flags & ZEND_ACC_FINAL)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could just be ZEND_ASSERT(prop_info) before this (only for IS_VALID_PROPERTY_OFFSET). The prop_info should be set if a valid property reference was null, if I understand how this works correctly.

I'd guess it'd be more efficient as UNEXPECTED(prop_info->flags & ZEND_ACC_FINAL && Z_PROP_FLAG_P(variable_ptr) != IS_PROP_UNINIT), because ACC_FINAL would probably be much less common than being initialized.

#define IS_VALID_PROPERTY_OFFSET(offset)           ((intptr_t)(offset) > 0)
#define IS_WRONG_PROPERTY_OFFSET(offset)           ((intptr_t)(offset) == 0)
#define IS_DYNAMIC_PROPERTY_OFFSET(offset)         ((intptr_t)(offset) < 0)

Copy link
Member Author

@kocsismate kocsismate Mar 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change the order of the condition, thank you for the pointing this out!

However, it seems to me that it's not possible to add a ZEND_ASSERT(prop_info) because as far as I saw, prop_info is only set for typed properties. And the if (UNEXPECTED(prop_info)) condition some lines below also suggests this I think.

echo $exception->getMessage() . "\n";
}

$foo->property2 ??= null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could document in the source of the test case that this is the first assignment to property2, and the test chose null because ??= would do nothing if the property value was not null and not initialized - I assume others may be as confused as you.

Same for the tests of static.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do! Actually, I was already considering it before, but then changed my mind.

@TysonAndre
Copy link
Contributor

Other edge cases that might need extra code in the unserializer, or warrant tests (I'd definitely think they'd be fixable if they have bugs):

  • Malformed data which sets the final property twice should throw (probably already handled?) (O:7:"MyClass":2:{s:9:"finalProp";i:2;s:9:"finalProp";i:3;})

    An annoying edge case would be unserializing properties that already have defaults - maybe those objects shouldn't throw at all (e.g. in __wakeup or __unserialize) until unserialization is over.

  • The unserializer creating final properties as references (e.g. due to being serialized with an older class definition which didn't use final)
    'O:7:"MyClass":2:{s:9:"finalProp";i:2;s:7:"refProp";R:2;}' for $x->refProp = &$x->finalProp;

I can imagine serialize() being used with final properties (e.g. to load from the database once and cache in memcache).

I'd naively guess the below snippet might throw an exception the way this is currently implemented, and if it doesn't, it'd still be useful to have the test to ensure behavior doesn't accidentally change in the future.

<?php

class X {
    final public $hasDefault = ['v7'];
}
$x = new X();
$s = serialize($x);
// Desired: This should not throw.
var_dump(unserialize($x));

(I maintain https://github.com/igbinary/igbinary , a compact binary serializer for php)

@kocsismate
Copy link
Member Author

kocsismate commented Mar 14, 2020

@TysonAndre I've started to add tests for serialization. I can say that

  • Malformed data which sets the final property doesn't throw an exception (yet)
  • Final properties that have a default value can be unserialized

I'll continue tomorrow :)

@kocsismate
Copy link
Member Author

@TysonAndre Today's findings:

The unserializer creating final properties as references (e.g. due to being serialized with an older class definition which didn't use final)

A serialized data like that is currently possible to unserialize. However, these properties can be currently modified via reference. :/.

Also, are there tests of references to fields of properties (e.g. what is the intended behavior for preg_match('/abc/', 'abcdef', $x->prop['field'])) - I skimmed the tests and didn't see any

Currently, an exception is thrown in this case with a message of "Cannot modify final property Foo::$property1 after initialization". I added a test for this case.

@TysonAndre
Copy link
Contributor

A serialized data like that is currently possible to unserialize. However, these properties can be currently modified via reference. :/.

ext/standard/var_unserializer.re has a check for whether it's possible to assign a value to a typed property while unserializing - I assume a similar error should be thrown if it was write-once. Not completely familiar with this part of the code.

		if (UNEXPECTED(info)) {
			if (!zend_verify_prop_assignable_by_ref(info, data, /* strict */ 1)) {
				zval_ptr_dtor(data);
				ZVAL_UNDEF(data);
				zval_dtor(&key);
				goto failure;
			}
			if (Z_ISREF_P(data)) {
				ZEND_REF_ADD_TYPE_SOURCE(Z_REF_P(data), info);
			}
		}

@@ -6396,19 +6398,18 @@ void zend_compile_prop_decl(zend_ast *ast, zend_ast *type_ast, uint32_t flags) /
"Property %s::$%s cannot have type %s",
ZSTR_VAL(ce->name), ZSTR_VAL(name), ZSTR_VAL(str));
}
} else if (flags & ZEND_ACC_FINAL) {
zend_error_noreturn(E_COMPILE_ERROR,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this part of the RFC. https://wiki.php.net/rfc/write_once_properties#compile-type_restrictions

Sadly, there's no way to write final public resource $fileHandle;, because resource isn't a type declaration in PHP (to give the flexibility to switch resources to objects).

I'd wonder if it'd be possible to remove the syntactic restriction internally (for write-once properties) and represent final as a property type info with "every type" if there is no explicit type, i.e. null|bool|int|float|string|object|array|<resource> (MAY_BE_RESOURCE) - I don't know if that's actually easy to do, though.

  • The alternative I have has its own drawbacks of implicitly and unexpectedly using the uninitialized state of typed properties, so enforcing manually adding a type is probably the best approach for now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something similar has already been discussed on the mailing list before. That is, implicitly treating all non-typed properties to be mixed. I think it would be much simpler and more straightforward to just wait for the mixed type instead. (I really cheer for it to reach PHP 8).

@@ -209,7 +209,7 @@ typedef struct _zend_oparray_context {
/* Static method or property | | | */
#define ZEND_ACC_STATIC (1 << 4) /* | X | X | */
/* | | | */
/* Final class or method | | | */
/* Final class, property, or method | | | */
#define ZEND_ACC_FINAL (1 << 5) /* X | X | | */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the visual representation in the table as well?

/* Class, property and method flags                  class|meth.|prop.|const*/	
Suggested change
#define ZEND_ACC_FINAL (1 << 5) /* X | X | | */
#define ZEND_ACC_FINAL (1 << 5) /* X | X | X | */

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather not update it for now because a new flag will be added for sure if the RFC is accepted (unless it's ok if I use the readonly keyword with ZEND_ACC_FINAL - but I think it's not).

#define ZEND_FETCH_OBJ_FLAGS 3
#define ZEND_FETCH_REF 1
#define ZEND_FETCH_DIM_WRITE 2
#define ZEND_FETCH_DIM_UNSET_FLAG 3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is deliberately the same value as ZEND_FETCH_OBJ_FLAGS, document the reason why?

Copy link
Member Author

@kocsismate kocsismate Mar 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is just because the flag that was removed from the same place a while ago had the same value. I'll have another shot with this flag since I couldn't make it work. :)

@mvorisek
Copy link
Contributor

mvorisek commented Apr 20, 2020

@kocsismate The RFC seems to be not accepted, do you know why and do you plan a revision/revote?

@kocsismate
Copy link
Member Author

kocsismate commented Apr 20, 2020

@mvorisek I believe these were the main reasons against: some people would prefer property accessors, and some people disliked the fact that cloning doesn't play well with readonly properties.

I'll certainly try to revisit this for PHP 8.1. :)

@psfpro
Copy link

psfpro commented Nov 30, 2020

Old school style

<?php
class Point {
    protected int $x;
    protected int $y;

    public function __construct(int $x, int $y = 0) {
        $this->x = $x;
        $this->y = $y;
    }
}

PHP 8.0.0 style

<?php
class Point {
    public function __construct(protected int $x, protected int $y = 0) {
    }
}

This is all good but I still need to write getters. And I'm disappointed that I don't see properties in the class description

<?php
class Point {
    public function __construct(protected int $x, protected int $y = 0) {
    }
    public function getX(): int
    {
        return $this->x;
    }
    public function getY(): int
    {
        return $this->y;
    }
}

That would be a great option for example if I want to use a class for DTO

<?php
class Point {
    readonly public int $x;
    readonly public int $y;
}

Example

<?php
class Point {
    readonly public int $x;
    readonly public int $y;
}

$point = new Point();
$point->x = 1;
$point->y = 1;

$arr = (array)$point;
echo json_encode($arr);

@kocsismate I will be glad to see this RFC in 8.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants