G-β - Python & Jupyter

How to install packages from the Jupyter Notebook safely

  • pip installs in any environment
  • conda installs conda environments
  • the shell environment and the Python executables are disconnected
  • Anaconda pre-install assumed below.
 
import sys
!conda install --yes --prefix {sys.prefix} <package>

See further discussion here.

How to enable Python debugger capabilities in Jupyter

  • Install the kernel xeus-python (new conda env or base env)
  • (It works fine with new fresh conda-env create/activate workflow)
  • Install debugger extension
 
conda install xeus-python notebook -c conda-forge
jupyter labextension install @jupyterlab/debugger

See further discussion here and here.

Learning resources

Journals

  • Journal of Open Source Software
    • Journal that is bursting on open source, counting with renowned people involved with machine learning, data science, and torrents of projects using Jupyter NBs.

Autograding

  • Autograding tools for testing Jupyter or making assignments easier to handle
    • nbgrader by Jupyter team.
    • autograde
    • Otter-Grader @Berkeley
    • Interesting report talking about graders and the alternative Web Cat.
    • Security failure in nbgrader is alleged and a solution is proposed by the Finnish SciComp group Aalto here.
    • This video shows nicely how to use grader with Github Classroom and accompanies this handout.

Lecturing with RISE

  • Notes on RISE Slideshow troubleshooting:
    • RISE stopped working correctly on Safari with Big Sur 11.2.3 suddenly.
    • Go to fix…
    • With version 5.7.1, I had an issue with
 
	pkg_resources.ContextualVersionConflict: 
	(jedi 0.15.2 ... Requirement.parse('jedi>=0.16'), {'ipython'})
  • With conda update jedi; conda list jedi, the version was not being upgraded to 0.18 (available on PyPi);
  • Worked only with conda update -c conda-forge jedi=0.17.2 after a long wait on “frozen/flexible solve… solving environment…”;
  • Then, launching Jupyter on Safari, the Cell Toolbar > Slideshow could be visible, but the button to launch RISE Slideshow was not showing up.
    • Current solution: use Chrome (see info in this page on how to set browser on Jupyter config file).
    • Persisting, try jupyter-nbextension install rise --py --sys-prefix and button should (luckly) appear!
  • How to customize RISE’s chalkboard and themes
    • List of available themes here.
    • From version 5.7.1, colors are customizable for chalkboard.
    • Options can be found in the reveal.js-plugins/chalkboard README
    • Here is an example on how to change chalkboard options:
 
"rise": {
  "chalkboard": {
      "boardmarkerWidth": 3,      
      "grid": {
        "color": "rgb(50,50,10,0.5)",
        "distance": 80,
        "width": 2
      }
    },
 }

How to edit the font color of a postscripted PDF

  • I will report a pathway although I have not gotten to a final solution yet:
  • Stefano Chizzolini, PDFClown’s developer proposed a code in this post. At the current version, however, PDFClown possibly has embedded all those isolated Java elements.
  • The idea would be try to follow SC’s approach.
  • Secondly, PyPDF2 might be a Python alternative. There is a tutorial here, but font color changes seems to be a hard stuff.
    • Why I wanted that? Modify color of latexized PDF text from black to CMY-balanced (after K=0) color to fill out CMY printer cartridges.
    • Let us leave that for future…

Python ‘gotchas’

Encoding/decoding characters

  • To print symbols for musical notes it is necessary to have a font in the computer capable to represent the Unicode. See here.
  • The Unicode table for music is here.
  • So, to print the G clef, we’d use print('\U0001D120'), with Unicode \U escape. Seemingly, there are few font types that handle music symbols. One available is Bravura.

Set browser for Jupyter Notebook or Jupyter-Lab

  • To choose a different browser from the computer’s default where to run Jupyter-Lab, run jupyter server --generate-config (or jupyter-notebook --generate-config) and edit the option c.ServerApp.browser in $HOME/.jupyter/jupyter_server_config.py
 
 #  Default: ''
import webbrowser
c.ServerApp.browser = u'/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome %s'
  • The module webbrowser allows us to choose among many known browsers. See table here.
  • For further detail, see this discussion.

PDF parsing end editing

  • pdfminer.six. Install with pip install pdfminer.six and run with pdf2txt.py file.pdf (install adds to PATH).
  • PyPDF2: good option.
  • WeasyPrint: conversion from HTML/CSS to PDF.
  • ReportLab
  • pyFPDF: ported from PHP.
  • Pyppeteer: JavaScript chrome/chromium browser automation library.
  • PDFKit: Javascript for Node and browser.

DataViz Tools

CSV to MD conversion

 
from mdtable import MDTable 
MDTable('input.csv').save_table('output.csv')

How to solve ModuleNotFoundError in VSCode while using Python Debugger

  • If using a virtual environment, such as a conda env, check which package manager is called. This is frequently caused by pip installs when the pip called is not the one from inside the env. Then, check if which pip points to the current environment. Otherwise, use python -m pip install <package_name> to execute the “true” pip. Then, use pip list to check if the package was installed.
  • Use Cmd + Shift + P and switch to the Python interpreter for your env.
  • Add "env": { "PYTHONPATH": "${workspaceRoot}"} to the configuration file launch.json.
  • Do it all and restart VSCode.