Add support for widgets in JupyterLab code consoles #3004

jtpio · 2020-10-30T19:53:44Z

This mimics the same approach as for the notebooks for now, but it might be possible to have both the consoles and notebooks share more logic.

jasongrout · 2020-10-30T20:52:13Z

Awesome!

Just curious, if a notebook and console share a kernel, do the widgets stay in sync?

jtpio · 2020-10-30T22:08:58Z

Just curious, if a notebook and console share a kernel, do the widgets stay in sync?

Not for now:

jtpio · 2020-11-02T14:14:22Z

It should be possible to make it work by changing the way widget managers are stored here:

ipywidgets/packages/jupyterlab-manager/src/plugin.ts

Line 112 in 5132234

let wManager = Private.widgetManagerProperty.get(context);

ipywidgets/packages/jupyterlab-manager/src/plugin.ts

Lines 323 to 330 in 5132234

    
             export const widgetManagerProperty = new AttachedProperty< 
        
               DocumentRegistry.Context, 
        
               WidgetManager | undefined 
        
             >({ 
        
               name: 'widgetManager', 
        
               create: (owner: DocumentRegistry.Context): undefined => undefined 
        
             }); 
        
           }

So that instead of a DocumentRegistry.Context it would be something specific to a kernel (or something common to both a notebook and a code console).

jtpio · 2020-11-02T14:36:56Z

It should be possible to make it work by changing the way widget managers are stored here:

Trying that out locally. for example by using the session context path (shared between the notebook and the console):

jasongrout · 2020-11-02T14:50:46Z

Nice! And it still works between two views of a notebook, or two views of an output?

jtpio · 2020-11-02T16:48:18Z

In that case yes, as all the views share the same WidgetManager instance:

for example by using the session context path

But using the path is a bit too brittle. Maybe there should be a SessionContextManager or similar that could be leveraged for both the consoles and notebooks main area widgets. Or checking whether a manager already exists for a notebook when creating a new console.

jasongrout · 2020-11-02T17:32:57Z

But using the path is a bit too brittle

Agreed. I haven't checked, but is there any object that the two KernelConnections share? Some reference to the underlying kernel representation, perhaps?

jasongrout · 2020-11-02T17:33:45Z

Or checking whether a manager already exists for a notebook when creating a new console.

I'd rather avoid having the console know too much about the notebook, and vice versa, as hard-coding these checks doesn't scale to other extensions as well.

tacaswell · 2021-03-12T03:00:50Z

🥜 gallery here: I am very excited about the PR and wanted exactly this with ipympl figures today.

caenrigen · 2021-03-12T12:47:40Z

Great PR, looking forward to this feature as well!! Kind regards

jtpio · 2021-03-15T12:21:27Z

Rebased so this branch can be tested on Binder:

jtpio · 2021-07-08T14:01:44Z

Just rebased one onto the latest main.

Trying that out locally. for example by using the session context path (shared between the notebook and the console):

Here is a diff to use the session context path:

diff --git a/jupyterlab_widgets/package.json b/jupyterlab_widgets/package.json
index 9f22f8cb..e4095418 100644
--- a/jupyterlab_widgets/package.json
+++ b/jupyterlab_widgets/package.json
@@ -68,7 +68,6 @@
     "@lumino/coreutils": "^1.3.0",
     "@lumino/disposable": "^1.1.1",
     "@lumino/messaging": "^1.2.1",
-    "@lumino/properties": "^1.1.0",
     "@lumino/signaling": "^1.2.0",
     "@lumino/widgets": "^1.3.0",
     "@types/backbone": "1.4.1",
diff --git a/jupyterlab_widgets/src/plugin.ts b/jupyterlab_widgets/src/plugin.ts
index 10fafc8d..fe5bb93b 100644
--- a/jupyterlab_widgets/src/plugin.ts
+++ b/jupyterlab_widgets/src/plugin.ts
@@ -33,8 +33,6 @@ import { toArray, filter } from '@lumino/algorithm';

 import { DisposableDelegate } from '@lumino/disposable';

-import { AttachedProperty } from '@lumino/properties';
-
 import { WidgetRenderer } from './renderer';

 import {
@@ -89,8 +87,7 @@ function* consoleWidgetRenderers(
 ): Generator<WidgetRenderer, void, unknown> {
   for (const cell of toArray(console.cells)) {
     if (cell.model.type === 'code') {
-      for (const codecell of ((cell as unknown) as CodeCell).outputArea
-        .widgets) {
+      for (const codecell of (cell as unknown as CodeCell).outputArea.widgets) {
         for (const output of toArray(codecell.children())) {
           if (output instanceof WidgetRenderer) {
             yield output;
@@ -136,11 +133,11 @@ export function registerWidgetManager(
   rendermime: IRenderMimeRegistry,
   renderers: IterableIterator<WidgetRenderer>
 ): DisposableDelegate {
-  let wManager = Private.widgetManagerProperty.get(context);
+  let wManager = Private.widgetManagerProperty.get(context.sessionContext.path);
   if (!wManager) {
     wManager = new WidgetManager(context, rendermime, SETTINGS);
     WIDGET_REGISTRY.forEach((data) => wManager!.register(data));
-    Private.widgetManagerProperty.set(context, wManager);
+    Private.widgetManagerProperty.set(context.sessionContext.path, wManager);
   }

   for (const r of renderers) {
@@ -172,14 +169,14 @@ export function registerConsoleWidgetManager(
   rendermime: IRenderMimeRegistry,
   renderers: IterableIterator<WidgetRenderer>
 ): DisposableDelegate {
-  let wManager = Private.widgetManagerProperty.get(console);
+  let wManager = Private.widgetManagerProperty.get(console.sessionContext.path);
   if (!wManager) {
     wManager = new KernelWidgetManager(
       console.sessionContext.session?.kernel!,
       rendermime
     );
     WIDGET_REGISTRY.forEach((data) => wManager!.register(data));
-    Private.widgetManagerProperty.set(console, wManager);
+    Private.widgetManagerProperty.set(console.sessionContext.path, wManager);
   }

   for (const r of renderers) {
@@ -249,7 +246,9 @@ function activateWidgetExtension(
       return;
     }

-    const wManager = Private.widgetManagerProperty.get(nb.context);
+    const wManager = Private.widgetManagerProperty.get(
+      nb.context.sessionContext.path
+    );
     if (wManager) {
       wManager.onUnhandledIOPubMessage.connect(
         (
@@ -414,12 +413,8 @@ namespace Private {
   /**
    * A private attached property for a widget manager.
    */
-  export const widgetManagerProperty = new AttachedProperty<
-    DocumentRegistry.Context | CodeConsole,
+  export const widgetManagerProperty = new Map<
+    string,
     WidgetManager | KernelWidgetManager | undefined
-  >({
-    name: 'widgetManager',
-    create: (owner: DocumentRegistry.Context | CodeConsole): undefined =>
-      undefined,
-  });
+  >();
 }
diff --git a/yarn.lock b/yarn.lock
index bae0d729..49a69346 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -1551,7 +1551,7 @@
     "@lumino/disposable" "^1.7.0"
     "@lumino/signaling" "^1.7.0"

-"@lumino/properties@^1.1.0", "@lumino/properties@^1.2.3", "@lumino/properties@^1.5.0":
+"@lumino/properties@^1.2.3", "@lumino/properties@^1.5.0":
   version "1.5.0"
   resolved "https://registry.npmjs.org/@lumino/properties/-/properties-1.5.0.tgz#7e8638e84c51bb110c5a69f91ca8b0e40b2c3fca"
   integrity sha512-YqpJE6/1Wkjrie0E+ypu+yzd55B5RlvKYMnQs3Ox+SrJsnNBhA6Oj44EhVf8SUTuHgn1t/mm+LvbswKN5RM4+g==

cc @trungleduc who was interested in this

trungleduc · 2021-07-09T08:02:24Z

Thank @jtpio, i'm trying to use kernel id from session context as map keys with handler for kernelChanged event. It works but is it a good approach?

jtpio · 2021-07-09T12:46:59Z

Thanks @trungleduc for looking into it.

kernel id from session context

Sounds like this could work? How does it behave with kernel restarts?

trungleduc · 2021-07-09T13:12:45Z

It works pretty well with kernel restart and kernel change events. I stored current id of kernel so inside the kernelChanged event handler i can update the map with new kernel id.

    sessionContext.kernelChanged.connect((_, args) => {
      const { newValue } = args;
      if (newValue) {
        const newKernelId = newValue.id;
        const oldwManager = Private.widgetManagerProperty.get(currentOwner);

        if (oldwManager) {
          Private.widgetManagerProperty.delete(currentOwner);
          Private.widgetManagerProperty.set(newKernelId, oldwManager);
        }
        currentOwner = newKernelId;
      }
    });

jtpio · 2021-07-12T07:31:00Z

NIce, thanks @trungleduc for sharing.

Maybe you would like to open a PR with your additional commits, so it's easier to check the diff and try it on Binder?

trungleduc · 2021-07-12T08:57:29Z

PR opened on jtpio#2

jtpio · 2021-07-19T13:03:53Z

Thanks @trungleduc!

I just pushed a change to fix the integrity check.

jtpio · 2021-07-19T13:13:21Z

For those who would like to test this PR on Binder:

https://mybinder.org/v2/gh/jtpio/ipywidgets/code-consoles-widgets?urlpath=lab

widgets-code-consoles.mp4

Marking as ready for review, so other folks can start testing the feature and reviewing the code. Thanks!

jtpio · 2021-07-23T08:47:03Z

jupyterlab_widgets/src/plugin.ts

+  sessionContext: ISessionContext
+): Promise<Private.IWidgetManagerOwner> {
+  await sessionContext.ready;
+  return sessionContext.session!.kernel!.id;


Wondering whether we should use the non-null assertion operator here?

A SessionContext can exist without having a kernel yet.

Wondering whether we should use the non-null assertion operator here?

~~I think it makes more sense to abort if we don't have a kernel. When we eventually get a kernel in the session context, the kernel changed signal is triggered which calls this again.~~

Update: Never mind. I've thought more about it, and put some thoughts below.

jtpio · 2021-07-23T08:51:10Z

Adding to the 8.0 milestone for consideration, since this is changing a couple of interfaces.

SylvainCorlay · 2021-07-23T22:35:58Z

This is great! Many thanks to @jtpio and @trungleduc!

I can't wait to see this land in the next 8.0 pre-release.

martinRenou · 2022-02-03T15:00:57Z

Looks good! This is really exciting, especially in combination with your recent work on replite

@jasongrout this PR is already a good improvement, maybe we could iterate later (in separate PRs) on making this more generic?

I don't want to be asking too much, but it would be amazing if this could be included in 8.0.

martinRenou

Looks good!

martinRenou · 2022-02-03T15:06:40Z

This seems to need a rebase though

jtpio · 2022-02-03T15:59:21Z

Thanks @martinRenou.

Starting the rebase now.

jtpio · 2022-02-03T17:06:31Z

Arf looks like the rebase and auto yarn resolve brings some new dependencies and typing conflicts:

Will get back to it later unless someone wants to give it a try before.

jtpio · 2022-02-07T08:08:12Z

For reference this means widgets also show up in the RetroLab (future Notebook v7) code consoles:

Temporary - it will properly work on future ipywidgets release see jupyter-widgets/ipywidgets#3004)

AckslD · 2022-10-31T14:04:29Z

I wanted to give this PR a try to see if I can help with testing, rebasing or anything. However when I do

pip install jupyterlab
./dev-install.sh

in a fresh virtualenv I still get the "Loading widget..." when creating eg an IntSlider. Are there any additional steps to make this work or has there been changes in eg jupyterlab what requires updates here?

davidbrochart · 2023-03-29T14:12:15Z

This PR is probably a big improvement for lots of reasons, but I have the feeling that consoles should not be a different beast than notebooks anyway. They should be a "restricted notebook", where one can only append new cells and not be able to change previous cells. If we based consoles on notebooks, then they would support widgets out of the box, right?

martinRenou · 2024-05-23T10:50:10Z

I just squashed to make the rebase easier (so that I don't need to fix the conflicts for all commits)

martinRenou · 2024-05-23T13:05:22Z

Tested with a custom widget, this seems to work like a charm:

martinRenou · 2024-05-23T13:14:00Z

This is changing exported APIs, we should do some exported functions renaming in order to stay backward compatible (if we don't want to wait for 9.x)

martinRenou · 2024-05-23T14:30:03Z

Docs build failure should be fixed by #3918

Co-authored-by: Duc Trung LE <[email protected]>

jtpio mentioned this pull request Jan 6, 2021

Make INotebookTracker optional in the manager plugin #3033

Merged

jtpio force-pushed the code-consoles-widgets branch from dd26cff to 9d2e0b2 Compare March 5, 2021 10:21

jtpio force-pushed the code-consoles-widgets branch from 9d2e0b2 to e7f6389 Compare March 15, 2021 12:07

jtpio force-pushed the code-consoles-widgets branch from e7f6389 to 5193ee4 Compare July 8, 2021 13:52

jtpio mentioned this pull request Jul 12, 2021

Use kernel id as keys of the map used to store widget manager jtpio/ipywidgets#2

Merged

jtpio marked this pull request as ready for review July 19, 2021 13:13

jtpio commented Jul 23, 2021

View reviewed changes

jtpio added this to the 8.0 milestone Jul 23, 2021

jasongrout self-requested a review August 10, 2021 17:17

martinRenou approved these changes Feb 3, 2022

View reviewed changes

jtpio force-pushed the code-consoles-widgets branch from 3b09ef1 to 016560f Compare February 7, 2022 08:06

jtpio mentioned this pull request Feb 9, 2022

2022 Jupyter Widgets Dev Meetings jupyter-widgets/team-compass#13

Open

bollwyvl mentioned this pull request Feb 14, 2022

Add a new repl app jupyterlite/jupyterlite#498

Merged

9 tasks

martinRenou mentioned this pull request Feb 17, 2022

How to make this work in Jupyterlab Code Console? matplotlib/ipympl#429

Closed

martinRenou mentioned this pull request Feb 25, 2022

Create docs matplotlib/ipympl#434

Merged

JoaoFelipe added a commit to EPICLab/DSChatbot that referenced this pull request Apr 8, 2022

Add executable code cell for ipywidgets output

0abc886

Temporary - it will properly work on future ipywidgets release see jupyter-widgets/ipywidgets#3004)

fleming79 mentioned this pull request May 17, 2024

Draft: Add bi-directional comms and run operations in tasks jtpio/ipylab#135

Closed

martinRenou force-pushed the code-consoles-widgets branch from 016560f to e642a78 Compare May 23, 2024 10:49

martinRenou force-pushed the code-consoles-widgets branch from e642a78 to 802fce2 Compare May 23, 2024 11:06

jtpio and others added 3 commits May 23, 2024 17:00

Widgets support in JupyterLab console

aaea376

Co-authored-by: Duc Trung LE <[email protected]>

toArray -> Array.from

c0c1cae

Backward compatibility

a66d362

martinRenou force-pushed the code-consoles-widgets branch from 573dc82 to a66d362 Compare May 23, 2024 15:00

trungleduc approved these changes May 23, 2024

View reviewed changes

martinRenou merged commit b12dae7 into jupyter-widgets:main May 24, 2024
20 checks passed

fleming79 mentioned this pull request May 25, 2024

DRAFT: Single widget manager per kernel #3920

Closed

jtpio deleted the code-consoles-widgets branch May 28, 2024 06:01

jtpio mentioned this pull request Sep 20, 2024

Broken ipywidgets interactivity in the Lorenz demo jupyter/try-jupyter#54

Open

Add support for widgets in JupyterLab code consoles #3004

Add support for widgets in JupyterLab code consoles #3004

Conversation

jtpio commented Oct 30, 2020

jasongrout commented Oct 30, 2020

jtpio commented Oct 30, 2020

jtpio commented Nov 2, 2020 • edited Loading

jtpio commented Nov 2, 2020

jasongrout commented Nov 2, 2020

jtpio commented Nov 2, 2020

jasongrout commented Nov 2, 2020

jasongrout commented Nov 2, 2020

tacaswell commented Mar 12, 2021

caenrigen commented Mar 12, 2021

jtpio commented Mar 15, 2021

jtpio commented Jul 8, 2021

trungleduc commented Jul 9, 2021

jtpio commented Jul 9, 2021

trungleduc commented Jul 9, 2021

jtpio commented Jul 12, 2021

trungleduc commented Jul 12, 2021

jtpio commented Jul 19, 2021

jtpio commented Jul 19, 2021

jtpio Jul 23, 2021

Choose a reason for hiding this comment

jtpio Jul 23, 2021

Choose a reason for hiding this comment

jasongrout Aug 19, 2021 • edited Loading

Choose a reason for hiding this comment

jtpio commented Jul 23, 2021

SylvainCorlay commented Jul 23, 2021

martinRenou commented Feb 3, 2022

martinRenou left a comment

Choose a reason for hiding this comment

martinRenou commented Feb 3, 2022

jtpio commented Feb 3, 2022

jtpio commented Feb 3, 2022

jtpio commented Feb 7, 2022

AckslD commented Oct 31, 2022

davidbrochart commented Mar 29, 2023

martinRenou commented May 23, 2024

martinRenou commented May 23, 2024

martinRenou commented May 23, 2024

martinRenou commented May 23, 2024

jtpio commented Nov 2, 2020 •

edited

Loading

jasongrout Aug 19, 2021 •

edited

Loading