bootstrap: include v8 and fs modules in the startup snapshot#36943

joyeecheung · 2021-01-15T14:16:56Z

The basic idea is:

Before serialization is done, iterate over supported embeder objects and perform whatever necessary to clean up the embedder objects. In the case of V8 and fs binding data, since we can discard the contents of these AliasedBuffers anyway, we'll just release them so that V8 don't error on unregistered global handles - I have a different POC of actually storing them behind private symbols so that we can retrieve them at deserialization time, but in these two particular cases it doesn't seem to worth the trouble, we can just reinitialize these buffers.
At serialization time, we save whatever data we need for each embedder slot to be deserialized properly. For v8 and fs bindings we just need to return a type constant in the callback for V8
At deserialization time, we enqueue functions in the deserialization callback of the context. We don't invoke them right away because it's more convenient to dehydrate the objects when the graph is completed (also potentially it's better if different embedder fields of one object depend on each other and need to be deserialized in a particular order).

src: support serialization of binding data

This patch adds the SnapshotableObject interface. Native objects
supporting serialization can inherit from it, implementing
PrepareForSerialization(), Serialize() and Deserialize()
to control how the native states should be serialized and
deserialized.

See doc: https://docs.google.com/document/d/15bu038I36oILq5t4Qju1sS2nKudVB6NSGWz00oD48Q8/edit

bootstrap: include fs module into the builtin snapshot

bootstrap: include v8 module into the builtin snapshot

nodejs-github-bot · 2021-01-15T14:36:32Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35560/

joyeecheung · 2021-01-15T14:58:41Z

CI is failing because the fs modules aren't yet vetted for runtime-dependence (so e.g. isWindows is cached into the snapshot across different builds). I'll vet these modules and open a separate PR to clean these up.

joyeecheung · 2021-01-28T16:42:47Z

I have split the first few commits into #37111, #37112, #37113, #37114

So that the debugger does not have to hard-code the number of internal fields of BaseObjects. PR-URL: #37111 Refs: #36943 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]>

This patch 1. Refactors the bootstrap routine of the main instance so that when --no-node-snapshot is used, Environment::InitializeMainContext() will only be called once (previously it would be called twice, which was harmless for now but not ideal). 2. Mark the number of BaseObjects in RunBootstrapping() when creating the Environment from scratch and in InitializeMainContext() when the Environment is deserialized. Previously the marking was done in the Environment constructor and InitializeMainContext() respectively for the cctest which was incorrect because the cctest never uses an Environment that's not bootstrapped. Also renames the mark to base_object_created_after_bootstrap to reflect what it's intended for. PR-URL: #37113 Refs: #36943 Reviewed-By: James M Snell <[email protected]>

So that it's easier to find the corresponding code. PR-URL: #37114 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

Previously, this was a per-class string constant for BindingData which is used as keys for identifying these objects in the binding data map. These are just type names of the BindingData. This patch renames the variable to type_name so that we can generalize this constant for other BaseObjects and use it for debugging and logging the types of other BaseObjects. PR-URL: #37112 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

1. Put the v8 binding data class into a header so we can reuse the class definition during deserialization. 2. Put the v8 binding code into node::v8_utils namespace for clarity. 3. Move the binding data property initialization into its constructor so that we can reuse it during deserialization 4. Reorder the v8 binding initialization so that we don't unnecessarily initialize the properties in a loop PR-URL: #37112 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

nodejs-github-bot · 2021-02-09T10:52:37Z

CI: https://ci.nodejs.org/job/node-test-pull-request/36033/

joyeecheung · 2021-02-09T14:07:00Z

cc @nodejs/startup This is now ready for review

This patch 1. Refactors the bootstrap routine of the main instance so that when --no-node-snapshot is used, Environment::InitializeMainContext() will only be called once (previously it would be called twice, which was harmless for now but not ideal). 2. Mark the number of BaseObjects in RunBootstrapping() when creating the Environment from scratch and in InitializeMainContext() when the Environment is deserialized. Previously the marking was done in the Environment constructor and InitializeMainContext() respectively for the cctest which was incorrect because the cctest never uses an Environment that's not bootstrapped. Also renames the mark to base_object_created_after_bootstrap to reflect what it's intended for. PR-URL: #37113 Refs: #36943 Reviewed-By: James M Snell <[email protected]>

So that it's easier to find the corresponding code. PR-URL: #37114 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

Previously, this was a per-class string constant for BindingData which is used as keys for identifying these objects in the binding data map. These are just type names of the BindingData. This patch renames the variable to type_name so that we can generalize this constant for other BaseObjects and use it for debugging and logging the types of other BaseObjects. PR-URL: #37112 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

1. Put the v8 binding data class into a header so we can reuse the class definition during deserialization. 2. Put the v8 binding code into node::v8_utils namespace for clarity. 3. Move the binding data property initialization into its constructor so that we can reuse it during deserialization 4. Reorder the v8 binding initialization so that we don't unnecessarily initialize the properties in a loop PR-URL: #37112 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

This patch adds the SnapshotableObject interface. Native objects supporting serialization can inherit from it, implementing PrepareForSerialization(), Serialize() and Deserialize() to control how the native states should be serialized and deserialized. See doc: https://docs.google.com/document/d/15bu038I36oILq5t4Qju1sS2nKudVB6NSGWz00oD48Q8/edit

nodejs-github-bot · 2021-02-19T09:24:43Z

CI: https://ci.nodejs.org/job/node-test-pull-request/36235/

This patch adds the SnapshotableObject interface. Native objects supporting serialization can inherit from it, implementing PrepareForSerialization(), Serialize() and Deserialize() to control how the native states should be serialized and deserialized. See doc: https://docs.google.com/document/d/15bu038I36oILq5t4Qju1sS2nKudVB6NSGWz00oD48Q8/edit PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

joyeecheung · 2021-02-19T11:13:11Z

Landed in 05286b9...445108d

aduh95 · 2021-02-21T14:43:56Z

It seems this PR broke the ASAN CI:

================================================================= LD_LIBRARY_PATH=/home/runner/work/node/node/out/Release/lib.host:/home/runner/work/node/node/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/runner/work/node/node/out/Release/obj/gen; "/home/runner/work/node/node/out/Release/node_mksnapshot" "/home/runner/work/node/node/out/Release/obj/gen/node_snapshot.cc" ==1174==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs operator delete []) on 0x60200002a2f0 LD_LIBRARY_PATH=/home/runner/work/node/node/out/Release/lib.host:/home/runner/work/node/node/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/runner/work/node/node/out/Release/obj/gen; "/home/runner/work/node/node/out/Release/mkcodecache" "/home/runner/work/node/node/out/Release/obj/gen/node_code_cache.cc" #0 0xa7d3fd in operator delete[](void*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0xa7d3fd) node.target.mk:26: recipe for target '/home/runner/work/node/node/out/Release/obj/gen/node_snapshot.cc' failed #1 0x34f7426 in v8::internal::ContextSerializer::SerializeJSObjectWithEmbedderFields(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f7426) rm 2ad06b2c9baade6b27c5c82cdcc7af11c369378d.intermediate 5e4cf980d3dc346b9b9ba01305d21285c5aecbf4.intermediate 9ac559b3661ef3e3ae7143f36ccc6691e578b31c.intermediate 02a486ff0aa87df511c38b125409357fdc559df2.intermediate #2 0x34f58ff in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f58ff) #3 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #4 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #5 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #6 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #7 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #8 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #9 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #10 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #11 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #12 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #13 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #14 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #15 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #16 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #17 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #18 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #19 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #20 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #21 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #22 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #23 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #24 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #25 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #26 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #27 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #28 0x2985568 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2985568) #29 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #30 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #31 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #32 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #33 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #34 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #35 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #36 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #37 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #38 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #39 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #40 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #41 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #42 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #43 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #44 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #45 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #46 0x2985eb9 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2985eb9) #47 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #48 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #49 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #50 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #51 0x2cdc92d in v8::internal::Serializer::SerializeRootObject(v8::internal::FullObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdc92d) #52 0x2cdc65a in v8::internal::Serializer::VisitRootPointers(v8::internal::Root, char const*, v8::internal::FullObjectSlot, v8::internal::FullObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdc65a) #53 0x34f4c46 in v8::internal::ContextSerializer::Serialize(v8::internal::Context*, v8::internal::PerThreadAssertScopeDebugOnly<(v8::internal::PerThreadAssertType)0, false> const&) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f4c46) #54 0x2cece90 in v8::internal::Snapshot::Create(v8::internal::Isolate*, std::vector<v8::internal::Context, std::allocator<v8::internal::Context> >*, std::vector<v8::SerializeInternalFieldsCallback, std::allocator<v8::SerializeInternalFieldsCallback> > const&, v8::internal::PerThreadAssertScopeDebugOnly<(v8::internal::PerThreadAssertType)0, false> const&, v8::base::Flags<v8::internal::Snapshot::SerializerFlag, int>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cece90) #55 0x204d41c in v8::SnapshotCreator::CreateBlob(v8::SnapshotCreator::FunctionCodeHandling) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x204d41c) #56 0x1090fff in node::SnapshotBuilder::Generate(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x1090fff) #57 0x108e67d in main (/home/runner/work/node/node/out/Release/node_mksnapshot+0x108e67d) #58 0x7fccc570bbf6 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21bf6) #59 0x9d5459 in _start (/home/runner/work/node/node/out/Release/node_mksnapshot+0x9d5459) 0x60200002a2f0 is located 0 bytes inside of 16-byte region [0x60200002a2f0,0x60200002a300) allocated by thread T0 here: #0 0xa7ca9d in operator new(unsigned long) (/home/runner/work/node/node/out/Release/node_mksnapshot+0xa7ca9d) #1 0x14632e6 in node::v8_utils::BindingData::Serialize(int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x14632e6) #2 0x1420d6b in node::SerializeNodeContextInternalFields(v8::Local<v8::Object>, int, void*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x1420d6b) #3 0x34f6a02 in v8::internal::ContextSerializer::SerializeJSObjectWithEmbedderFields(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f6a02) #4 0x34f58ff in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f58ff) #5 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #6 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #7 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #8 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #9 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #10 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #11 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #12 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #13 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #14 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #15 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #16 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #17 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #18 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #19 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #20 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #21 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #22 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #23 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) #24 0x2984889 in void v8::internal::BodyDescriptorApply<v8::internal::CallIterateBody, void, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*>(v8::internal::InstanceType, v8::internal::Map, v8::internal::HeapObject, int, v8::internal::ObjectVisitor*) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2984889) #25 0x2ce1c6a in v8::internal::Serializer::ObjectSerializer::SerializeContent(v8::internal::Map, int) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce1c6a) #26 0x2cdfc4e in v8::internal::Serializer::ObjectSerializer::SerializeObject() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2cdfc4e) #27 0x2ce16c7 in v8::internal::Serializer::ObjectSerializer::Serialize() (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce16c7) #28 0x34f5d6d in v8::internal::ContextSerializer::SerializeObjectImpl(v8::internal::Handle<v8::internal::HeapObject>) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x34f5d6d) #29 0x2ce39d3 in v8::internal::Serializer::ObjectSerializer::VisitPointers(v8::internal::HeapObject, v8::internal::FullMaybeObjectSlot, v8::internal::FullMaybeObjectSlot) (/home/runner/work/node/node/out/Release/node_mksnapshot+0x2ce39d3) SUMMARY: AddressSanitizer: alloc-dealloc-mismatch (/home/runner/work/node/node/out/Release/node_mksnapshot+0xa7d3fd) in operator delete[](void*) ==1174==HINT: if you don't care about these errors you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0 ==1174==ABORTING make[2]: *** [/home/runner/work/node/node/out/Release/obj/gen/node_snapshot.cc] Error 1 make[1]: *** [node] Error 2 make: *** [build-ci] Error 2 Makefile:104: recipe for target 'node' failed Makefile:530: recipe for target 'build-ci' failed Error: Process completed with exit code 2.

GH Actions is failing consistently since this landed on master.

RaisinTen · 2021-02-21T14:55:04Z

@aduh95 fixed here: #37443

This patch adds the SnapshotableObject interface. Native objects supporting serialization can inherit from it, implementing PrepareForSerialization(), Serialize() and Deserialize() to control how the native states should be serialized and deserialized. See doc: https://docs.google.com/document/d/15bu038I36oILq5t4Qju1sS2nKudVB6NSGWz00oD48Q8/edit PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

So that the debugger does not have to hard-code the number of internal fields of BaseObjects. PR-URL: #37111 Refs: #36943 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]>

Previously, this was a per-class string constant for BindingData which is used as keys for identifying these objects in the binding data map. These are just type names of the BindingData. This patch renames the variable to type_name so that we can generalize this constant for other BaseObjects and use it for debugging and logging the types of other BaseObjects. PR-URL: #37112 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

1. Put the v8 binding data class into a header so we can reuse the class definition during deserialization. 2. Put the v8 binding code into node::v8_utils namespace for clarity. 3. Move the binding data property initialization into its constructor so that we can reuse it during deserialization 4. Reorder the v8 binding initialization so that we don't unnecessarily initialize the properties in a loop PR-URL: #37112 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Jan 15, 2021

joyeecheung force-pushed the baseobj-reinit branch from 831d67d to 75283c6Compare January 15, 2021 14:36

joyeecheung force-pushed the baseobj-reinit branch from 75283c6 to e9d9076Compare January 28, 2021 12:22

This was referenced Jan 28, 2021
src: refactor binding data for deserialization support #37112
Closed
src: refactor bookkeeping of bootstrap status #37113
Closed
src: put (de)serialization code into node_snapshotable.h/cc #37114
Closed

joyeecheung force-pushed the baseobj-reinit branch from e9d9076 to 357cd1dCompare January 28, 2021 16:33

joyeecheung mentioned this pull request Jan 28, 2021
src: expose BaseObject::kInternalFieldCount in post-mortem metadata #37111
Closed

joyeecheung force-pushed the baseobj-reinit branch from 357cd1d to 451c7a7Compare January 28, 2021 16:41

joyeecheung added a commit that referenced this pull request Feb 5, 2021
src: put (de)serialization code into node_snapshotable.h/cc
2e769a9
So that it's easier to find the corresponding code. PR-URL: #37114 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

joyeecheung force-pushed the baseobj-reinit branch from 451c7a7 to bb64b7dCompare February 8, 2021 16:33

joyeecheung marked this pull request as ready for review February 9, 2021 14:01

joyeecheung changed the title ~~WIP: support v8 and fs BindingData in the startup snapshot~~bootstrap: support v8 and fs BindingData in the startup snapshotFeb 9, 2021

joyeecheung changed the title ~~bootstrap: support v8 and fs BindingData in the startup snapshot~~bootstrap: include v8 and fs modules in the startup snapshotFeb 9, 2021

jasnell approved these changes Feb 14, 2021
View reviewed changes

danielleadams pushed a commit that referenced this pull request Feb 16, 2021
src: put (de)serialization code into node_snapshotable.h/cc
337b4e7
So that it's easier to find the corresponding code. PR-URL: #37114 Refs: #36943 Reviewed-By: Juan José Arboleda <[email protected]>

joyeecheung added 3 commits February 19, 2021 16:57

bootstrap: include fs module into the builtin snapshot
ed19c32

bootstrap: include v8 module into the builtin snapshot
8da45ff

joyeecheung force-pushed the baseobj-reinit branch from bb64b7d to 8da45ffCompare February 19, 2021 09:23

joyeecheung added a commit that referenced this pull request Feb 19, 2021
bootstrap: include fs module into the builtin snapshot
8aa9b77
PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

joyeecheung added a commit that referenced this pull request Feb 19, 2021
bootstrap: include v8 module into the builtin snapshot
445108d
PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

joyeecheung closed this Feb 19, 2021

joyeecheung mentioned this pull request Feb 25, 2021
Node.js Technical Steering Committee (TSC) Meeting 2021-02-25 nodejs/TSC#972
Closed

targos pushed a commit that referenced this pull request Feb 28, 2021
bootstrap: include fs module into the builtin snapshot
59861ba
PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

targos pushed a commit that referenced this pull request Feb 28, 2021
bootstrap: include v8 module into the builtin snapshot
d2e5529
PR-URL: #36943Fixes: #35930 Refs: #35711 Reviewed-By: James M Snell <[email protected]>

targos mentioned this pull request Mar 2, 2021
v15.11.0 release proposal #37569
Merged

targos added the backport-blocked-v14.x label May 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

bootstrap: include v8 and fs modules in the startup snapshot#36943

bootstrap: include v8 and fs modules in the startup snapshot #36943

Uh oh!

joyeecheung commented Jan 15, 2021•
edited
Loading

Uh oh!

nodejs-github-bot commented Jan 15, 2021

Uh oh!

joyeecheung commented Jan 15, 2021

Uh oh!

joyeecheung commented Jan 28, 2021

Uh oh!

nodejs-github-bot commented Feb 9, 2021

Uh oh!

joyeecheung commented Feb 9, 2021

Uh oh!

nodejs-github-bot commented Feb 19, 2021

Uh oh!

joyeecheung commented Feb 19, 2021

Uh oh!

aduh95 commented Feb 21, 2021

Uh oh!

RaisinTen commented Feb 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

bootstrap: include v8 and fs modules in the startup snapshot#36943

bootstrap: include v8 and fs modules in the startup snapshot #36943

Uh oh!

Conversation

joyeecheung commented Jan 15, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

src: support serialization of binding data

bootstrap: include fs module into the builtin snapshot

bootstrap: include v8 module into the builtin snapshot

Uh oh!

nodejs-github-bot commented Jan 15, 2021

Uh oh!

joyeecheung commented Jan 15, 2021

Uh oh!

joyeecheung commented Jan 28, 2021

Uh oh!

nodejs-github-bot commented Feb 9, 2021

Uh oh!

joyeecheung commented Feb 9, 2021

Uh oh!

nodejs-github-bot commented Feb 19, 2021

Uh oh!

joyeecheung commented Feb 19, 2021

Uh oh!

aduh95 commented Feb 21, 2021

Uh oh!

RaisinTen commented Feb 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

joyeecheung commented Jan 15, 2021•
edited
Loading