Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Switch to spidermonkey/mozjs 78 #80
Comments
|
Yes, nothing new here. CJS needs to bump towards a new mozjs now and then. The next bump is likely to be towards mozjs60. CJS 4.4 is using mozjs52, that won't change. The bump could happen for 4.6 in the next development cycle or failing that in 4.8. |
|
By that time we should make it mozjs68 https://gitlab.gnome.org/GNOME/gjs/issues/270 |
mkg20001
changed the title
* Stolen from meta-oe right before mozjs was upgraded to 60.x * In the long term mozjs-52 can go because cinnamon intends to move to mozjs-68 [1]. That is not yet available in any layer as far sa I know (off-topic have started work on mozjs-68 to get rid of python2 but did not yet get it to fly) * build might fail on other arches than arm but that is good enough for me [1] linuxmint/cjs#80 Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
|
Hi @clefebvre Debian maintainer of Cinnamon here.
Do you have plans to implement the switch for 4.8? On the Debian side we are now phasing out the old mozjs and the switch is becoming a serious bug. Thanks for all your work Norbert |
|
An update from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=908234:
|
|
Update 2020-09
Any chance for an update here? |
|
#84 (thanks to leigh123linux) about CI failing is for not related error, @clefebvre can you take a look please?: |
|
I'm currently rebuilding cinnamon on nixOS with the PR, I'll post an update once I have some results |
I switched cinnamon for f34 to use gjs https://src.fedoraproject.org/rpms/cinnamon/commits/master The change to cjs was to keep it in stasis so it doesn't get retired, also a couple of third-party applets use cjs. The cjs port also requires some changes to cinnamon code https://leigh123linux.fedorapeople.org/pub/patches/new_cjs.patch |
|
FTR I believe @clefebvre intends to use mozjs68 for Mint. |
|
@leigh123linux I agree! |
|
I got some warnings during session start Besides that seems to work with @leigh123linux cjs & cinnamon patch |
|
I suppose that clem didn't switch to newer mozjs because want a mozjs present in both latest mint version, mint20 based on focal have mozjs 52 and 68, lmde 4 based on buster that have mozjs 52 and 60, then mozjs 52 is the only present on both. |
|
This one is new The ES6 standard warnings were present in the virgin code. |
|
@leigh123linux Ok, seems like it's just a warning tho for now, at least didn't see anything broken Also there's a PR for nixOS to use the patched version for the time being, so we are at least no longer depending on the insecure mozjs52 NixOS/nixpkgs#99008 |
I believe I've seen at least a 15% decrease in cinnamon startup time with mozjs78 using gjs or cjs. |
similar results here |
|
Let's just try to make the switch, I'll try to build with mozjs78 Clem, I don't know if you are staying with 52 because it's in Ubuntu Focal and you can put it in Mint Repos, but I suggest that everyone as a whole, who packages Cinnamon and uses it for any distros other than Mint, such as leigh123linux for Fedora, need to get this fixed ASAP before this becomes project suicide. |
mkg20001
changed the title
If that cjs patch is too large there is another path, use gjs instead. https://src.fedoraproject.org/rpms/cinnamon/blob/master/f/switch_to_gjs.patch |
|
remove a fork too little maintained it would be beautiful but unfortunately if cinnamon is not keep updated for the compatibility of gjs versions as soon as they come out there is a risk of more problems, so keep the fork (but upstream and not only single distros patches, for less waste of collective time and better results) I suppose it is necessary to have less risk of bugs/unexpected events, especially for rolling releases distro (or similar) |
|
Arch is not planning to remove either mozjs52 or cinnamon at the moment. I've been watching this issue for ~6 months and I'm happy to see there's movement to getting this resolved as I'd quite like to consolidate on one recent mozjs too. But I'd really like to see the PR merged before packaging it, and I'm disinclined to apply a distro-specific patch. Even if the patch is specific to more than one distro. This needs to be fixed properly rather than requiring every distro to maintain out of tree patches. |
|
What are the advantages and disadvantages of using gjs directly? If it's simple to do, I'm a bit curious why cinnamon ended up forking it to cjs back in the day... |
gjs used to break abi every release so cjs was forked to solve this, since then gjs change between releases seems to have slowed/stopped. |
Agree in principle, but distributors have to do something now, and cannot wait for whatever might come at an unspecified time in the future. So for now I think the update as mentioned is necessary, at least for some here. |
|
for cinnamon itself from what i have read so far it seems better switch to gjs. |
|
I did some tests: cinnamon using cjs with mozjs52 (debian unstable, same of upstream): cinnamon using updated cjs with mozjs78 (debian experimental, using patches from leigh123linux): cinnamon using gjs (deb https://www.preining.info/debian unstable cinnamon main, using patches from leigh123linux) and removing cjs: @clefebvre can you say something please? thanks for any reply and sorry for my bad english |
|
Thanks @Fantu for the detailed testing. I think the case that applets rely on |
|
This is just my opinion and not more, but as i mentioned more less the same in gjs, i think now is time to said it here. I think it is possible to continue trying to convince someone who apparently don't want to be convinced or instead probably the best is contribute to port cjs to the last gjs code, but that without to lost support for the Mint spidermonkey active version. In that way should be reduced the number of changes that are need as all of that was included with enough time in the master branch. That move will also help to port cjs to a new version of gjs when Mint decide, because what will be needed was already included in cjs and all package that depend of it like the spices (extensions). As it's important reduce the difference between cjs and gjs as possible, one way of how this could be done is if all different code between both can be assimilated by gjs and/or by cjs in the best way for both. Also, when something can not be assimilate in gjs or cjs, the best could be inject the difference in the Cinnamon shell instead of modify the cjs package. For example, this is used in Cinnamon to override some Gio functionalities that some GNOME developers don't want to change: But exactly this code is not needed anymore because gjs assimilated it and create a much robust solution than what it is in cjs actually. Unfortunately no one ported the robust solution of gjs back to cjs and this is the kind of things that shouldn't happen, because it is a thing that was already done and then it only create a difference between the two package just for nothing. There are a lot of change that are coming and they should be prevented with enough time: |
|
Hi everyone, Thanks for raising this issue and contributing to this discussion. Sorry for the time it took me to join you. We're going to prioritize this issue and resolve it ASAP and for everybody. I'd like to thank @ptomato also for his message and talk a little bit about GJS. CJS is not a "real" fork. It's a fork of course, but in terms of project, it has no ambition to be any different than GJS. None of the changes which were added to CJS were envisioned or planned when the fork first occurred, some of them were ported upstream, and those which weren't can be put in Cinnamon directly. CJS has also been rebased quite a few times on GJS. It doesn't deviate from it, it just catches up. So when it comes to "differences" between GJS and CJS, they don't matter to us per se. We don't need our own interpreter. What we need is our own "version" of the interpreter. Let me try and explain this a little bit, it's quite important. When a distribution ships a new version of GNOME it includes a version of GJS which is appropriate and which works with that version of GNOME. When it ships a new version of Cinnamon is ships with the CJS interpreter of the same version. In the past, when Cinnamon used GJS directly, new versions of Cinnamon were designed with the version of GJS used in Linux Mint and LMDE in mind. So although Cinnamon worked very well in these distributions, it could break in distributions which shipped newer or older versions of GJS, and these distributions could not upgrade or downgrade or freeze their version of GJS because that version was required by the version of the GNOME they shipped with. Even if CJS was 100% identical to GJS (and it could be), what it really brings to the table is this: It allows us to know that a particular version of Cinnamon runs with the same interpreter in all distributions. It doesn't matter who's running a brand new GJS and who's running a really old one, when the new Cinnamon comes out, there's a matching CJS and everybody can use that. Distributions can ship with GNOME, Cinnamon, GJS, CJS and two versions of mozjs, what they cannot do is ship with two versions of GJS. Now, in terms of our own targets, in addition to yours, we have the following:
I assume in rolling distributions we're looking at mozjs78 and gjs 1.66 and if we were to look at compatibility with the current LTS (thankfully we're not but it's relevant all the same) we'd be looking at mozjs52 and gjs 1.52. If we switch to GJS we become more flexible but we risk seeing issues and behavior which differ between distributions and which we might not always be able to reproduce or even fix for everybody. If we jump to mozjs78 we fix the issue for all distributions, old and new, and ensure everyone can run the next version of Cinnamon without running in specific issues. The problem then of course is that although that solves the problem for now, we'll be having the same problem again next year. Moving to GJS is a risk in terms of compatibility because of that. Even if GJS itself tries to minimize the impact on GNOME and Cinnamon, it's not just up to them, moving to a newer mozjs can mean mandatory code changes and unlike GNOME we're not just worried about having our latest version run on the latest mozjs, we'd have to juggle with both old and new here and work with different versions if we want to port the latest Cinnamon to LMDE/Debian and Mint/Ubuntu. With CJS the version of mozjs is the same everywhere, whether that's an old 52 or the brand new 78 and that allows us to run the upcoming Cinnamon anywhere we want. |
|
Just a brief comment: Although it's not really taking place yet in all distros, I believe the goal for GNOME developers is for gjs updates to be backported to LTS distributions. So both GNOME Shell and a Cinnamon-GJS would need updates to continue working with the supported version of GJS. This is because mozjs, as a part of Firefox, is only supported for about one year. |
Just a minor note; while this is a worthy goal, I do want to be clear that I and the other upstream GJS contributors don't have enough bandwidth to support this, so this is really up to the LTS distributions themselves. |
|
The other potential approach to ensuring every distro can use the same gjs/mozjs version for cinnamon, would be to make gjs capable of having multiple versions of gjs coexist and properly versioning itself based on semantic versioning + "when you need mandatory code changes". Currently, libgjs uses the soname "0.0.0" and every other file is installed to a "1.0" namespace. I'm not actually sure what the point of this "1.0" namespace is if it actually does break compatibility though. ;) One could then depend on old versions of vanilla gjs and package them just like, currently, multiple versions of mozjs get packaged. In spirit this is even what cjs is. |
Same here, we wouldn't be updating past Cinnamon releases to switch to newer versions of mozjs. That said, if newer versions of mozjs were backported in Debian stable and Ubuntu LTS we would definitely switch to them faster. |
|
@clefebvre |
|
Ok, I guess the backporting gjs thing is just a Ubuntu LTS idea that's been on a wishlist for 2 years and no one is working on it yet. |
|
hi @Fantu No, we're not waiting on anyone. We'll be fixing this ASAP in preparation for Cinnamon 4.8. We're testing both scenarios (sorry, I just can't get used to the Latin plural form). You're right, I prefer the CJS solution but I'm also interested in seeing if we can use GJS and even if we don't, if we can reduce the difference and make it easier to switch. Right now we're preparing mozjs78 builds for LMDE 4 and Mint 20. We'll then make a PR to port CJS (and possibly Cinnamon if code changes are needed) to it. |
|
@clefebvre big thanks for your work trying to solve it fast
|
|
@mtwebster Thank you for this commit, you're a saint getting us somewhere |
|
I don't think it's a good idea to be backporting stuff-mozjs52 is becoming ancient and with Focal being supported for 5 years and Mint following itself for 4 years with this, 4.8 use GJS. IMO best idea |
|
As a GJS contributor, I would love to see Cinnamon use GJS moving forward. I think it would unify the communities and reduce the maintenance burden for the GJS and CJS maintainers, distributors, and packagers but I understand the hesitancy and @clefebvre's point on versioning and stability difficulties. Ultimately, I hope Cinnamon finds the best solution for the Mint community. That said, I would just like to clarify any "API breakage" concerns and lay out how/in what circumstances GJS could be used. GJS will not have any API breaks until it reaches What can change, however, is our support for JS language features as SpiderMonkey evolves. Essentially two things can happen:
As JavaScript is now a rigorously standardized language SpiderMonkey does not, as a matter of policy, add non-standard features anymore. Most of the obvious non-standard features have since been removed. (the migration to 60 in particular removed a lot) If Cinnamon chooses to use GJS and a newer version of GJS loses support for a non-standard feature such as Non-Standard (removed a few years ago): String.toUpperCase('lowercase') // LOWERCASEStandard: 'lowercase'.toUpperCase() // LOWERCASECinnamon can issue a patch to use the standard language feature and it will continue working on both older and rolling versions of GJS. To use GJS you just have to avoid using new features until your lowest GJS version supports them and if you're using non-standard features you should fix the usages to maintain future and current compatibility. From what I know nothing has been removed without a standard implementation that works. You can read through the GJS release docs and look for |
|
Also @clefebvre thank you for your work regardless, I'm glad to see CJS moving toward |
|
@ewlsh I think that one idea to contribute with the downstream (not only for Cinnamon), is create a way for run gjs in compatibility mode with an older version. That could be implemented internally just adding something like an script file to be loaded only when gjs is running in compatibility mode with an older version. I will use your own example as an example to run gjs in compatibility mode with mozjs < 60. In that case we should probably add something like that: if (!String.hasOwnProperty("toUpperCase")) {
String.toUpperCase = function(text) {
return text.toUpperCase();
}
}This could be used to reduce the gap between the gjs versions and will let gjs to move on. Of course, this don't need to be perfectly because for example, i don't see how to implement in a good way a compatibility procedure to the case of the introduction of But I think that for most common things this should help a lot. Especially for cases like Cinnamon. |
|
For the if (Gio.File.prototype.load_contents) {
Gio._real_load_contents = Gio.File.prototype.load_contents;
Gio.File.prototype.load_contents = function(cancel) {
let contents = Gio._real_load_contents(cancel);
if (contents[1] instanceof Uint8Array) {
contents[1] = imports.byteArray.toString(contents[1]);
}
return contents;
};
}This code works instead. It's crappy, but the only way to go thanks to the gjs issue: if (Gio._LocalFilePrototype && Gio._LocalFilePrototype.load_contents) {
Gio._real_local_load_contents = Gio._LocalFilePrototype.load_contents;
Gio._LocalFilePrototype.load_contents = function(cancel, ...args) {
let contents = Gio.File.prototype.load_contents.call(this, cancel, ...args);
if (contents[1] instanceof Uint8Array) {
contents[1] = imports.byteArray.toString(contents[1]);
}
return contents;
};
} |
|
Hi everyone, I've got some good news. Mozjs78 is now in Mint and LMDE and we'll be merging support for it into the master branches of CJS and Cinnamon early next week. Some of the devs have been testing this for a little while now and it looks pretty stable. As an additional bonus we're observing slight performance improvements, in particular during the init phase. The new Cinnamon will adapt to the new JS syntax (load_contents/toString) but the spices will need to work with both old and new versions of Cinnamon. We had a few solutions in mind and one of them consisted in overriding load_contents. @mtwebster wasn't successful with trying that. Maybe @lestcape's code above will help. So far the chosen solution is simply to use spice multiversioning (https://projects.linuxmint.com/reference/git/cinnamon-tutorials/xlet-versioning.html). 15 of them are impacted, it's quite manageable. It's not the most elegant solution but it's a proper one, multiversioning was designed specifically for retro-compatibility, whether that's older versions of Cinnamon itself or the JS interpreter it uses. We'll also bump CJS and Cinnamon to unstable version 4.7 so we can implement proper dependencies in the packaging and multiversioning in the spices without having to wait for 4.8. |
|
@lestcape I tried the first one and it didn't work, I sort of figured it was because GFile was only an interface. I missed the local file patch somehow in the overrides file. This stinks this only addresses file:// uris but that covers probably 99% of any likely uses, I'll try that second approach, thanks. |
|
@mtwebster and @clefebvre I think only fourteen are impacted because this one is using a fairly common runtime check: const [, content] = file.load_contents(null);
let converted;
if (content instanceof Uint8Array) { // mozjs60 future-proofing
converted = ByteArray.toString(logs);
} else {
converted = logs.toString();
}The other ones will print this warning if you don't patch the environment: (gjs:62025): Gjs-WARNING **: 09:01:19.619: Some code called array.toString() on a Uint8Array instance. Previously this would have interpreted the bytes of the array as a string, but that is nonstandard. In the future this will return the bytes as comma-separated digits. For the time being, the old behavior has been preserved, but please fix your code anyway to explicitly call ByteArray.toString(array).
(Note that array.toString() may have been called implicitly.)But can be quite easily updated with the above check and they should still function normally (albeit with a warning until updated). Several spices seem to use If you are going to patch the environment I'd recommend a few modifications... if (Gio._LocalFilePrototype && Gio._LocalFilePrototype.load_contents) {
Gio._real_local_load_contents = Gio._LocalFilePrototype.load_contents;
Gio._LocalFilePrototype.load_contents = function(cancel, ...args) {
let contents = Gio.File.prototype.load_contents.call(this, cancel, ...args);
if (contents[1] instanceof Uint8Array) {
contents[1] = new ByteArray.ByteArray(contents[1]); // wrap it like this!
}
return contents;
};
}Otherwise the fix won't work if any spices attempt to use the contents as an actual ByteArray and you'll be passing a The only other usecase for Gio.File in an extension might be resources, which you could also patch. |
|
@mtwebster Yes the first don't work as i mention. The second suggestion works, but this was a suggestion for gjs, to support the older code. It can be used in the Cinnamon case to not need to update the code, but the patch recommend by @ewlsh i think could be better if works (but it don't work for me). |
|
Be advised that the underscore in |
|
@mtwebster I will add another possibility that Cinnamon have. This is port all the Cinnamon code related with const Gio = imports.gi.Gio;
const ByteArray = imports.byteArray;
if (!ByteArray.hasOwnProperty("toString")) {
// We can assume that we have an 'UTF-8' encoding like was in the older gjs
ByteArray.toString = function(data, encoding) {
return data.toString();
};
}
if (Gio._LocalFilePrototype && Gio._LocalFilePrototype.load_contents) {
Gio._real_local_load_contents = Gio._LocalFilePrototype.load_contents;
Gio._LocalFilePrototype.load_contents = function(cancel, ...args) {
let contents = Gio._real_local_load_contents.call(this, cancel, ...args);
// We can assume that this the older ByteArray implementation is not Uint8Array.
if (!(contents[1] instanceof Uint8Array)) {
// Wrap the simple text in a ByteArray to be possible use later ByteArray.toString
contents[1] = ByteArray.fromString(contents[1]);
}
return contents;
};
}
let data = Gio.File.new_for_path("testCaseApp.js").load_contents(null)[1];
// The data is ByteArray or Uint8Array and ByteArray.toString exist in both cases.
// So, we can do now always:
print(ByteArray.toString(data));In that way all the Cinnamon code belong to the last version of gjs and when you remove the monkey patch, you don' t need to change the code again. Also (probably a better idea) you can monkey patch the ByteArray.toString and nothing more, to add support for all the possibilities you have in the older and new gjs versions. This should be something like this: const Gio = imports.gi.Gio;
const ByteArray = imports.byteArray;
ByteArray._real_toString = (ByteArray.hasOwnProperty("toString")) ? ByteArray.toString : null;
ByteArray.toString = function(data, encoding) {
if ((data instanceof Uint8Array) && ByteArray.hasOwnProperty("_real_toString")) {
return ByteArray._real_toString(data);
}
if (data.hasOwnProperty("toString")) {
return data.toString();
}
return data;
};
let data = Gio.File.new_for_path("testCaseApp.js").load_contents(null)[1];
// ByteArray.toString exist and it don't care if data is ByteArray or Uint8Array or also a simple text,
// it will return an string of it always.
print(ByteArray.toString(data));Note: All that code is inspired in this gjs commit and are only tested in gjs 1.64.3 |
|
Mozjs78 support was merged today in CJS/Cinnamon. Both projects were bumped to unstable version 4.7. We'll probably modify the deprecation warning in Lines 53 to 67 in ddff379 |
|
Closing this, as I think this issue has been sufficiently addressed. |
The Firefox ESR 52 is no longer maintained, as such it likely contains unfixed security vulnerabilities.
Firefox ESR 60 is the next ESR, which is currently maintained.
Could CJS be updated to support mozjs60?