Expectation about tier of support from diagnostic tools and VMs

[joyeecheung]

Action items:

1. Look for feedback from users about their dependencies on existing diagnostic tools (e.g. do they block an upgrade of Node.js because tool X stops working on newer version of Node.js)
2. Look for feedback from the developers of the diagnostic tools about their support for Node.js updates (e.g. if there is going to be a change in Node.js that is going to break a tool, are they willing to support it and how long would that take)
3. Look for feedback from VM vendors about their support for the development of diagnostic tools (e.g. if they are going to break an existing tool, are they willing to provide support to work with the tools and unbreak those, what level of priority would that be)
4. Create a table (or tables) about different tier of support among different category of tools, Node.js core and the VM vendors

[mmarchini]

I made a non-exhaustive list of tools we have today (either inside Node.js core, in the VM or as external tools). If I forgot anything, feel free to edit the list and include it.

Tracing

OS/External Tracing Tools

• DTrace
• LTTng
• ETW
• SystemTap
• eBPF tracing tool

Other

• async_hooks
• V8 trace_events
• Node.js trace_events

Profiling

V8

• CPUProfiler
• HeapProfiler
• SamplingHeapProfiler

External Stack Samplers

• Linux perf
• eBPF profile tool
• DTrace
• Windows xperf

Heap and Memory Analysis

• mdb_v8
• node-heapdump
• Chrome DevTools
• llnode

Step Debugging

• Chrome Debugging Protocol
• V8 Debugging Protocol
• ChakraCore Time Travel Debugging

Other tools

• node-report
• 0x
• node-clinic
• eBPF node-specific tools (nodegc and nodestat)

[mmarchini]

Diagnostics WorkGroup Deep Dive Meeting on Expected Support Tiers - 2018-??-??

Time

TBD

Agenda

Brainstorm of available diagnostic tools

Before discussing support tiers for diagnostic tools, we need to know which
tools we have today and how stable they are.

Questions we need to answer

• Which relevant diagnostic tools we have today in the Node.js ecosystem.

Note: we might want to start a poll to gather feedback from users about this.
Maybe we should reach out to the User Feedback WG?

Support tiers for tools outside Node.js core

Most diagnostic tools we have today exist outside of Node.js core, which means
it's harder for us to keep them updated and working, even if they're under the
Node.js Foundation (i.e., llnode).

Questions we need to answer

• Should we have test suites in place for those tools?
  • If so, should they be integrated into our current test suite or should they
    be part of CITGM? (or even something else)
• Should we move some of these tools into core? (we're already doing it for
  node-report).

Node.js/V8 Support Tiers

Some tools rely on V8's non-supported APIs and features to work (i.e., Linux
perf, llnode, DTrace, 0x with --prof, etc.). Those tools are relevant in Node.js
context because they usually offer greater granularity and less overhead when
diagnosing an application (qualities important for i.e., servers but not so
relevant to browsers).

Questions we need to answer

• How to handle breakage of these tools?
  • Should we block V8 updates if some of these tools break?
  • Should we provide built-in alternatives to these tools?

Note: another big difference between Node.js and browser is the presence of
Native Modules on Node.js.

Discuss possible support tiers for diagnostic tools

Define support tiers with their respective expectations

Questions we need to answer

• Which support tiers should we provide?
• What are the expectations for each support tier?

Define criteria for tools to go under a specific support tier

Questions we need to answer

• What are the criteria for a tool to be under a specific support tier?
  • Tests? Number of contributors? Dependency on unsupported V8 APIs?

Invited

• Diagnostics team: @nodejs/diagnostics

[yunong]

Should we discuss this during the upcoming collaboration summit? There's already a session booked for diagnostics. cc @mhdawson

Otherwise, we (Netflix) would be happy to host a diagnostic summit at our offices in Los Gatos, CA at some point.

[mhdawson]

I think we should add to the topics we discuss at the collaborator's summit. It's also good to start planning for the next summit dedicated to diagnostics. The only thing I'd add on that front is that there was some discussion that maybe the next one should be in Europe?

[jasnell]

I can look into having nearForm host in Ireland if y'all would like :) maybe around NCEU so we can consolidate travel.

[davidmarkclements]

If not nearForm hosting, I think it would be awesome if the V8 team host at Google, Munich /cc @hashseed @bmeurer

[mhdawson]

I like the idea of consolidating travel with NCEU.

[yunong]

@mhdawson Given the conversation from the collaborator's summit, can we start to get alignment on tiers of support for the toolchain that @mmarchini has listed here?

[mhdawson]

@yunong lets start with a list of the tools and then we can build the matrix which answers the questions above for each tool.