Thanks both for answering. And so fast!
First, I tried your code, Posthoholic. It leaves me with the same problem as my own, I’m afraid. It is the Machine variable I care mostly about so that further PowerShell command windows will have an updated path. If I run both yours and mine code locally it works as expected. If I restart the computer or log in and out it also work as expected. If I only starts a new console, the paths are not updated when remoting is used for setting the path. So unluckily, it seems that Don Jones is right about this.
In the end I did a work-around calling SetX.exe from my PowerShell code locally, using the -s parameter to update a remote computer. That worked as expected.
Posthoholic, thanks for your better check. I have updated my code accordingly. I really liked the readability and compactness of that more correct but also more complex test.