nyblcore is…

• a tiny ATtiny85-based lo-fi sample player device for diy enthusiasts.
• powered by a single AAA for ~5-8 hours of battery life.
• holding 8 kB of storage for up to 1.2 seconds of 8-bit audio sampled @ 4.2 kHz.
• controlled with three multi-functional knobs that modulate tempo, volume, distortion, and fx.
• manipulated with three fx knobs for audio warping (jump, retrig, stutter and stretch).
• able to load custom firmware / audio using an Arduino or an AVR usb programmer.
• open-source*, wonderfully hackable.

demo

buy

unfortunately, we are currently sold out. devices will be available again on November 1st.

please sign up for the mailing list, and you will be notified as soon as devices are ready again:

Questions? Reach out at zack@infinitedigits.co or review the purchase policy.

examples

Example 1: drum hits!

original audio:

nyblcore:

Example 2: breakbeat!

original audio:

nyblcore:

Example 3: arp

original audio:

nyblcore:

Example 4: just a sine wave

original audio:

nyblcore:

documentation

full instructions are available here: download pdf

soldering tutorial

open-source

nyblcore is open-source. in addition, all the code and boards are on my github. If you use the public repo to build a nyblcore, I ask that you consider sponsoring me on Github to help me continue making devices and software that is open-source.

limitations = possibilities

The tiny size of nyblcore is both its strength and its limitation. The limited memory and storage space only allow for a small amount of audio and effects. Additionally, the tempos may not be entirely stable and there is no external sync, but this can also be seen as an opportunity to embrace the unpredictability and randomness of the device.

I hope that nyblcore may be a welcome challenge or a way to break out of creative ruts. or maybe it can serve as a source of inspiration for creating new sounds and effects.

upload

You can download the original firmware here. Or use this form to generate a new firmware with your audio for the nyblcore:

bytebeat

this is an alternative firmware that is compatible with the nyblcore. download the bytebeat firmware from github and upload it as you would normally (see instructions below).

in the bytebeat firmware the selector knob (middle) selects between two modes:

1. selector fully counter-clockwise: in this mode the leftmost knob (knob a) will change volume/distortion. the rightmost knob (knob b) will select a bytebeat formula from ~40 different formulas.
2. selector knob fully clockwise: in this mode the leftmost knob will select a parameter in the current bytebeat formula. the rightmost knob will change that parameter in the current bytebeat formula. changes are reset upon power-cycling.

Uploading firmware

If you are starting from scratch, skip this for now and read further below about the tools you need for the tools necessary to upload the firmware.

It is easy to upload a firmware obtained from the upload form. Simply copy the resulting code:

You can copy and paste this directly into the Arduino IDE. Then take out the ATtiny85 from the nyblcore board and put it in your programmer and plug your programmer into your computer. Now just hit the upload button on Arduino to upload the new firmware and voila!

You can take out the programmer and put the newly loaded ATtiny85 back into the nyblcore board to enjoy the new sounds.

Uploading firmware tools

Changing the audio on the ATtiny85 requires uploading new firmware. Uploading new firmware is easy but it requires a USB ATtiny85 programmer. (You can also use an Arduino, see the instructions for programming, though make sure to use 16 MHz not 8 when selecting the speed.).

Windows users: you will also need drivers for the USB ATtiny85 programmer.

To start, make sure you install the Arduino IDE. Open the software and goto File -> Preferences and where it says Additional Boards Manager URLs you can click a button to add an additional line. Add the following line:

https://raw.githubusercontent.com/damellis/attiny/ide-1.6.x-boards-manager/package_damellis_attiny_index.json

Now restart the Arduino IDE. Now you can goto Tools -> Board: -> ATtiny Microcontrollers and select ATtiny25/45/85.

Before you begin you will need one of the following pieces of hardware. You can either purchase a USB ATtiny85 programmer (like this one) OR

To upload new firmware, first make sure that you install the Arduino IDE. Any version will work.

Next, you will need to select Tools -> Clock -> Internal 16 Mhz.

Finally, you will need to select Tools -> Programmer -> USBtinyISP.

That’s it! If you run into trouble feel free to send me a message at zack@infinitedigits.co .

Happy audio warping!

pikocore is…

• a lo-fi music mangler based on the Raspberry Pi Pico (brother of the nyblcore).
• capable of holding 8 minutes of 8-bit 33 kHz monophonic samples.
• powered by a single AAA battery for up to 3 hours or by powered by USB-C.
• tempo-synced with a selectable BPM between 60 and 300, with samples mangled by beat-synced effects (stutter, retrig, gate, tunneling).
• loaded with real-time effects like a resonant filter, timestretching, volume, and wavefolding.
• sequenced with a 128-step sequencer with recording/playback
• saved and loaded via EEPROM for instant patch recall.
• able to load custom firmware, new samples, all through USB-C.
• sync-compatible with Pocket Operators.
• open-source, wonderfully hackable.

for more info, check out my interview with Studio Brootle and a review by DJ Iceman.

demo

original audio:

pikocore:

buy

in addition to my store, you can buy assembled devices from Perfect Circuit: click here to go to Perfect Circuit. also, if you are in the UK, diy kits are available from Thonk: click here to go to Thonk.

sign up for the mailing list to get notified about updates and new things.

cases

Pikocore cases are now available for the DIY pikocore, sold separately through Dich Studios. Features: Frosted buttons which allow the LEDs to shine through, Full access to the battery, usb and audio ports. A matching power switch cap for easy operation, Rubber feet to minimize sliding, and several slots for attaching a lanyard.

usage

full instructions are available here: download pdf

bom

there is an alternative build guide developed from noise.kitchen available here: download pdf

tutorial

basics

there are eight buttons and four knobs that can be used to manipulate the pikocore. there are two outputs and one input. make sure to plug in only stereo cables to any outputs or inputs.

i/o

the top left jack is for incoming trigger signal (e.g. from a pocket operator in SYN1 mode or a pikocore). the top right jack sends out audio. the bottom right jack sends out trigger signal and audio signal (for syncing to pocket operators using SYN4 mode).

buttons

The buttons on the device can be used to jump to samples and retrigger audio in real time.

jumping: Press any of the eight buttons to jump to that relative position in the sample.

retriggers: Press and hold a button and press another button to create a retrigger effect. The retrigger speed is based on the position of the second button press. Random effects add added to the retriggers, such as changes in speed, pitch, filter envelopes, or volume envelopes.

stop/play: press the two leftmost buttons and the two rightmost buttons simultaneously to stop/play.

Button mashing is encouraged.

knobs

The rightmost knob is a filter/volume knob that controls the output level. The leftmost knob is the selector knob, which can choose between eight different positions. Each position changes a different pair of parameters, which can be changed using the middle two knobs, knob A and knob B.

• sample changes the sample being played (holds >100 samples totaling ~8 minutes).
• break modifies all probabilities simultaneously using varied easing functions.
• filter is a resonant low-pass filter that attenuates higher frequencies.
• stretch performs a lofi timestretch effect.
• gate controls the amount of gating on the sample.
• gate prob. controls the probability of gating.
• jump prob. controls the likelihood of jumping to a different step in the current sample.
• retrig prob. controls the likehood of retriggering.
• tunnel prob. controls the likelihood of jumping to a different sample.
• reverse prob. controls the probability of reversing direction.
• sequencer rec will record sequences, up to 128 steps (cw = record, ccw = erase).
• sequencer on turns on the sequencer (cw = on, ccw = off).
• save saves probabilities, sample, volume, and tempo (cw = save).
• load recalls the last save (cw = load).
• volume/fold changes the volume and adds a wavefolding effect.
• tempo controls the tempo in steps of 5 bpm (50-305), encoded in binary.

syncing

pikocore uses a audio click signal to sync and is fully compatible with sync signals coming from other devices, like the teenage engineering pocket operators. The pikocore syncing operates at 2 PPQN and has max input/output voltages of 3V.

The upper left audio jack takes in a PO sync signal (SYN1 on Pocket Operator) and will automatically sync the pikocore. The output sync is separate from the main audio output jack, situated below the Pico mcu. When using a pocket operator, set the receiving device to SYN4 (or SYN5 if the chain continues.

clock locking (available in version v1.2+)

you can “lock” the clock so that the beat continually stays on track and doesn’t get diverted when you jump between samples. this is super helpful if you are syncing and you don’t want to get “off” beat when jumping between audio.

you can activate/deactivate this feature by pressing the middle four buttons (yellow = on, teal = off).

more information in this video:

tempo

Set the tempo by turning the selector knob fully CW to the eighth spot and then turning knob b. The BPM available is between 50 and 305, and the current value is displayed using binary encoding. Instead of memorizing the binary encoding, you can use tool below to visualize the tempo you want to match. Change the slider and then on the pikocore move knob b so that the lights match up to get this bpm. Because of the clock in the pikocore, BPMs are only accurate to 2%.

midi

the pikocore is capable of listening to midi using a itty bitty midi device plugged into its clock input.

this behavior needs to be enabled at the firmware level, and the firmware tool below can be used to update the settings to listen to midi.

the pikocore automatically sets tempo and syncs to a midi clock. it also listens to midi start/stop/continue commmands to synchronize with sequencers. the firmware tool lets you change the midi clock division. you can also toggle a setting in the firmware to have the key presses of a midi keyboard work as buttons, so you can sequence your own button presses.

update

Updating the pikocore is easy. Simply press the “boot” button and plug in a USB-C cable, or press and hold “boot” while pressing “rst” when the USB-C cable is already plugged into your computer.

Then download the .uf2 file (either from factory reset firmware, or customized firmware, see below) and drag-and-drop it to the drive that pops up in your computer (usually named RPI-RP2). Visual instructions are available with this YouTube video that explains the process for uploading firmware.

factory reset

firmware

download these to get the pikocore back to the “stock” image or upgrade it to a newer image.

• Feb 2, 2024: v1.2.2 (fix saving) .uf2 w/ midi enabled, .uf2 w/ clock enabled
• Dec 2nd, 2023: pikocore-73bf07d.uf2 (v1.1.0, midi enabled)
• Sept 6th, 2023: pikocore-ad319a8.uf2 (v1.0.1, clock enabled, no RGB led)
• July 4th, 2023: pikocore-1ca6156.uf2 (v1.0.1, clock enabled)

customized firmware

Use this tool to change the audio on the pikocore. This will automatically utilize the latest firmware of the pikocore.

Uploaded audio is automatically converted to the base sample rate (33 kHz) and to the base BPM (165 BPM). If you know the BPM of your sample, you can help the server detect it by including it in the filename somewhere as bpmX. For example, my_drum_bpm150.wav will automatically be registered as 150 BPM. You can also include the number of beats in the filename somewhere as beatsY. For example, my_drum_bpm150_beats4.wav will be registered as having 4 beats. If no information is included, the server will do its best to detect the BPM and then determine the number of beats to be used for beat syncing.

alternative firmware

there are some great alternative firmwares that utilize the same hardware but with a different solution!

pikobeats

a drum sequencer for the pikocore.

by @rheslip! source code: https://github.com/rheslip/PikoBeats

piko-33

a sample sequencer for the Pikocore.

also by @rheslip! source code: https://github.com/rheslip/Piko-33

remixing

(water color image by Sebtic Smile)

send me your remixes! they light up my day (literally!). there are some amazing ingenuity with creating enclosures for pikocore:

• arcade box! and button box by 2ononen
• arcade box by ground_grown_circuits
• printable case w/ 3D step files available (regular case, expanded case)

troubleshooting

Here are some fixes for issues that may crop up when using pikocore.

The audio is really quiet. This happens if you plug in a mono cable to the audio output. Make sure to use only 3.5mm stereo cables and make sure they are pluggined in all the way.

There is a high pitched sound. This seems to happen sometimes while powered by the AAA battery and the potentiometers are in a certain position. Try turning the potentiometers CCW to eliminate the noise. If that doesn’t work, try rotating the battery in is holder. If it is still a problem, try powering from a USB powerbank.

There is no sound, or the sound is muffled sounding. This can happen if the filter gets set fully closed, or the timestretch is at 100%, or the volume if fully down. The best thing to do is to reset these three parameters. Move the selector knob to the second position and turn knob a fully CW and turn knob b fully CCW. This will reset the filter and the timestretch. Now move the selector knob to the eight and last position and try to turn knob a until you can hear something (the volume). This should fix it 90% of the time.

If you still have issues: Please feel free to send me an email at zack@infinitedigits.co.

itty bitty midi a simple circuit that takes either a MIDI din input or a MIDI trs type-a input, and converts it to a logic-level serial bit stream. the resulting bit stream can be used by *core devices (like the pikocore) to listen to MIDI signals directly through the clock inputs.

there are two versions of this product, select the one that suits you: the active version utilizes a battery-powered circuit with the 6n138 optocoupler to effectively isolate the downstream and upstream devices. the passive version does not use isolation and has no battery.

the itty bitty midi is open-source.

buy

demo

here is a little demo of me synchronizing the output of the tb03 with the pikocore using the midi out from the tb03 into the itty bitty midi, into the clock input of the pikocore.

source

it is easy and encouraged to DIY the itty bitty midi. both the passive and active versions are easy to make with throughhole components, but the passive one is by far the easiest if you have a few capactiors and a DIN plug.

getting started

the usage of itty bitty midi is simple enough: connect your midi device “midi out” with midi din cable (5-pin) or a midi trs type a cable to the itty bitty midi.

then connect a stereo cable from the output of the itty bitty midi (the headphone jack facing away from the 5-pin din jack) to the clock input of a pikocore or zeptocore.

usage

once connected, the device will listen to incoming midi. in the case of the pikocore, it will listen to midi start/stop/continue commands to start & stop on beat. the pikocore also listens to the midi timing commands and automically synchronizes. for more information, see the pikocore midi help.

windows magic

on windows, you no longer need to install TDM-GCC or MinGW-w64 to develop against CGo. Using scoop you can simply install zig:

1scoop install zig

and then your CGo build will just work with the following command:

1$env:CGO_ENABLED=1; $env:CC="zig cc"; go build -v -x

I was having so many troubles with weird errors in MinGW-w64¹ and this just worked.

linux magic

the linux situation is even more incredible.

one of the common things I want to do is cross-compile for windows from linux. zig also makes this super easy with -target:

1CGO_ENABLED=1 CC="zig cc -target x86_64-windows-gnu" \
2    GOOS=windows GOARCH=amd64 go build -v -x

even more magical is the Mac OSX situation, which was terribly riddled with strange errors and requires it’s own SDK. luckily, Jose Quintana is hosting MacOS X SDK’s which can be used to compile against Mac OS X in a simple Makefile incantation:

 1build: MacOSX11.3.sdk
 2	MACOS_MIN_VER=11.3 MACOS_SDK_PATH=$(PWD)/MacOSX11.3.sdk \
 3        CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 \
 4	CGO_LDFLAGS="-mmacosx-version-min=$${MACOS_MIN_VER} --sysroot $${MACOS_SDK_PATH} -F/System/Library/Frameworks -L/usr/lib" \
 5	CC="zig cc -target aarch64-macos -isysroot $${MACOS_SDK_PATH} -iwithsysroot /usr/include -iframeworkwithsysroot /System/Library/Frameworks" \
 6	go build -ldflags "-s -w" -buildmode=pie -v -x 
 7
 8MacOSX11.3.sdk:
 9	wget https://github.com/joseluisq/macosx-sdks/releases/download/11.3/MacOSX11.3.sdk.tar.xz
10	tar -xvf MacOSX11.3.sdk.tar.xz

the Mac OS X incantation itself is a little more involved because it has to grab the Frameworks library and I only learned about it from Luca Corbo’s post detailing it and other one-liners. but again, it just works.

github actions

the best for last - all of these tools work perfectly well in Github Actions. I am working on a project that is compiled for all OS’s, using CGo, and I now have a Github action that automatically builds new releases.

• a mid-fi music mangler based on the Raspberry Pi Pico (brother of the nyblcore and the pikocore).
• mono or stereo playback of 16-bit (internal 32-bit) audio files @ 44.1 kHz sampling rate
• powered by two AAA batteries for up to 4 hours or by powered by USB-C
• tempo-synced with a selectable BPM between 60 and 300.
• loaded with 16 real-time effects like timestretching, delay, distortion, fuzz, tape stop, resonant filters, etc..
• sync-compatible with any gear that has a clock input/output signal.
• open-source, wonderfully hackable.

buy

tool

demos

still a wip…

Orders are run through “Shopify”. The infinite digits store is part of my company “infinite digits” so when you make a purchase you will see credit card bills for “infinite digits” on your bills.

Taxes

International orders are subject to import taxes and duty upon arrival, which will be paid by the receiver. This is typically around 20% of the retail cost. The carrier will contact you when the item arrives in customs to arrange payment. If this payment is refused and the package is returned, we will be forced to pay this duty and return shipping, which will be deducted from your refund.

Cancellations

Orders which have not yet shipped may be canceled any time before shipment for a full refund minus a 2.6% + $0.30 (US) or 3.6% + $0.30 (international) processing fee. Refunds must be sent to the original form of payment.

Shipping & Delivery

We use “Shopify” to send both within USA and internationally. The price for shipping therefore varies depending on where in the world it is to be sent. This appears at check-out and a confirmation is also sent via email when the order is sent. In most cases, tracking of packages is included and you as a customer will then also receive a tracking link on the email you enter when purchasing. Orders are shipped within three business days.

Returns

Within 7 days of delivery, email zack@infinitedigits.co to arrange a return. If you’re in the US, I’ll email you a USPS return label. if you’re outside of the US, I’ll ask you to arrange return delivery using your preferred international shipping provider.

Re-pack items in their original packaging, as you received them. Once I receive the return, any item(s) in new/unused condition will receive a full refund (excluding any shipping costs and minus a 2.6% + $0.30 (for US) or 3.6% + $0.30 (for international) processing fee).

Stock

Devices are made in small batches and are built up in the capacity they are built. Therefore, only the units listed on the website are in the quantity listed on the website. The stock is updated with each completed order. If an item is on the page but is eventually out of stock, this means that it will be updated as soon as it is built. No pre-purchases can be made, units are only sold when they are ready and in stock.

Warranty

If your device is not working as intended, please feel free to contact me at zack@infinitedigits.co to swap the item or arrange a refund. Every device is backed by a one year warranty to cover manufacturing failure (shipping costs not covered).

If any device fails during regular use within one year of purchase by the original owner, please let me know as soon as it happens by emailing zack@infinitedigits.co. Be sure to include your order number and a description of the problem.

SD card in the audio block

In many applications (mine included), audio is transfered in “blocks”, i.e. the audio dac asks for a batch of samples all at once instead of one sample at a time. For real-time audio this is a good strategy to buffer playback so that a sample isn’t lost in case another sample takes too long to compute. In my audio application the audio block consists of 441 samples. This means, at a sampling rate of 44,100 samples/second (44.1 khz), there is only 10 milliseconds to generate the next piece of audio.

If the audio data is in RAM, there is almost no latency to access the data. However, when the audio is coming from the SD card, then there is some latency involved in sending messages to the SD card. To prevent dropouts (bad audio “pops”), the SD card must succesfully retrieve data in less than 10 milliseconds.

So my most basic question is “what is the best SD card to use” which revolves around two questions:

1. How fast can a SD card retrieve audio data in my application?
2. How often does the SD card require more than 10 milliseconds to retrieve data?

Experiment

I bought 14 different SD cards and tested them using the same firmware, same hardware, and same samples. My firmware uses the SDIO protocol, for this test it is accessing 819 bytes after seeking to a position. I use the rp2040 time_us_64() to get the timing before and after the operations, and add up the entire time to get the latency.

(click the image to enlarge). The box-plots show the median read duration (latency) in my application and the dots show the outliers. The red area at the top shows where latency exceeds the audio block time, causing dropouts. (code for making plot)

Experiment 2 (overclocking)

A few weeks later, I decided that I need to run my rp2040 at 250 Mhz instead of 125 Mhz to eek out much more CPU processing power. The SDIO does not run this fast, so I am increasing the clock divider from 1 to 2 so that the SDIO actually runs at the same speed.

(click the image to enlarge). In this case we see partially the same story - “Lexar” is fast but unreliable. But also now we can see that “Kootion” is fast and unreliable as well. However, interestingly, the “PNY” and the “SP Elite” chips which used to be unreliable are now more reliable and their speed matches most of the other cards. The slow card here is, again, the SanDisk Ultra 16GB, which is a bit old. Also the “Kingston 2GB” is apparently so old that the SDIO never worked on it at all.

Conclusion

I found that most SD cards are pretty fast. There are some that are much slower (an old Sandisk 16GB). Surprisingly, the fastest SD card (Lexar A1 32 GB) is not usable because it has periodic spikes of high latency which causes audio dropouts.

The best SD card then is somewhere in the middle - not too fast and mostly reliable!

not all SD cards are the same!

what is gno?

Gno isn’t just another programming language; it’s an interpreted version of the Go language, optimized for the intricacies of blockchain and deterministic execution on distributed systems. If you’re familiar with Go, transitioning to Gno is a breeze.

Here are some key differentiators between Go and Gno:

• Special Packages: Gno introduces specialized packages like std for accessing the caller’s address and avl.Tree for a deterministic map.
• Deterministic Design: Gno execution is verifiable and is capable of running on distributed systems.
• Transpilation Magic: Under the hood, Gno code is transpiled into Go, leveraging the robust Go compiler system.

The differences may seem minute, but they pave the way for powerful blockchain applications with unparalleled precision.

There is a lot more information on their main website at gno.land. Also you can just easily play with Gno code online too, at the play.gno.land website.

What is a Smart Contract?

“Smart contracts” are are immutable, self-executing programs living on the blockchain.

Beyond automating transactions, smart contracts can be used for various applications, such as creating incentivized social networks and reshaping interactions with the web (i.e., “web3”). Smart contracts written in Gno run within the Gno.land ecosystem.

What is Gno.land?

Gno.land is a platform for writing smart contracts in Gno, the first of a series of Gno Layer 1 chains. Built on Tendermint2, Cosmos/IBC, and secured by Proof of Contribution, Gno.land prioritizes simplicity, security, scalability, and transparency.

web3 Bytebeat Symphony

For my first attempt into web3 I wanted to make a simple, shareable system of music. Music could be encoded into the blockchain and it could be easily recorded (verifiably) that some piece of music was generated first by some identity on the chain. I went for bytebeat music because its very straightforward to code. bytebeat was discovered by viznut in 2011 as a way to type a very short computer programs that generate chiptune music, for example this is a bytebeat program:

1main(t){
2  for(;;t++) putchar(((t<<1)^((t<<1)+(t>>7)&t>>12)));
3}

which you can take the raw output of and convert to audio:

1gcc -o crowd crowd.c
2./crowd | head -c 4M > crowd.raw
3sox -r 8000 -c 1 -t u8 crowd.raw crowd.wav

simplicity, built-in

The thing that amazed me about the Gno language is how fast it is to get started. First of all, the language is basically Go, plus a few addons. Also its mostly “batteries included” (persistent globals = database, getting callers = identification). Here is, essentially, my entire “smart contract” for generating bytebeat music, in just 50 lines of code:

 1package bytebeat
 2
 3import (
 4	"std"
 5
 6	bytebeat "gno.land/p/demo/audio/bytebeat/v1"
 7	"gno.land/p/demo/ufmt"
 8)
 9
10type Comment struct {
11	User    string
12	Message string
13}
14
15var (
16	data     string
17	comments []Comment
18)
19
20func init() {
21	seconds := uint32(10)
22	data = bytebeat.ByteBeat(seconds, 8000, func(t int) int {
23		return (t>>10^t>>11)%5*((t>>14&3^t>>15&1)+1)*t%99 + ((3 + (t >> 14 & 3) - (t >> 16 & 1)) / 3 * t % 99 & 64)
24	})
25}
26
27func AddComment(s string) string {
28	caller := std.GetOrigCaller() // main
29	comments = append(comments, Comment{
30		User:    string(caller),
31		Message: s,
32	})
33	return "ok"
34}
35
36func Render(path string) string {
37	output := ""
38	output += "\n"
39	output += "# my bytebeat\n"
40	output += `<audio controls="controls" autobuffer="autobuffer" autoplay="autoplay">
41<source src="data:audio/wav;base64,`
42	output += data
43	output += `" />
44</audio>`
45	output += "\n\n## comments\n"
46	for i, comment := range comments {
47		output += ufmt.Sprintf("%d. *%s* - %s\n ", i, comment.Message, comment.User)
48	}
49	return output
50}

There are some clever built-in functions here, like the Render function is automatically used by gno.land to do web rendering - so in addition to a smart contract you get a web server for free.

This smart contract is not yet totally “in action” yet because there is no main net to deliver it. But it is really straightforward and easy to setup your own Gno.land server to run smart contracts. I wrote a whole presentation on the topic which you can read more about here.

Microblogging in the GnoVerse

For another attempt into web3 I was interested in making a Twitter-like thing - essentially a way to “microblog”. Again, my smart contract for this package is extremely concise:

 1package microblog
 2
 3import (
 4	"std"
 5
 6	"gno.land/p/demo/microblog"
 7	"gno.land/r/demo/users"
 8)
 9
10var (
11	title  = "gno-based microblog"
12	prefix = "/r/demo/microblog:"
13	m      *microblog.Microblog
14)
15
16func init() {
17	m = microblog.NewMicroblog(title, prefix)
18}
19
20// Render calls the microblog renderer
21func Render(path string) string {
22	return m.Render(path)
23}
24
25// NewPost takes a single argument (post markdown) and
26// adds a post to the address of the caller.
27func NewPost(text string) string {
28	if err := m.NewPost(text); err != nil {
29		return "unable to add new post"
30	}
31	return "added new post"
32}
33
34func Register(name, profile string) string {
35	caller := std.GetOrigCaller() // main
36	users.Register(caller, name, profile)
37	return "OK"
38}

A lot of work is being done behind the scenes here with some “packages”. There is a difference between “Packages” and “Realms” in Gno - where both are like Go packages in that they are modular and importable, but “Realms” have the special ability that they persist globals, as they are meant to be run on the block chain.

The package for the microblog is the most important and uses a lot of the cool Gno features. There is a AVL tree for managing data and some user management properties built into the language:

  1package microblog
  2
  3import (
  4	"errors"
  5	"sort"
  6	"std"
  7	"strings"
  8	"time"
  9
 10	"gno.land/p/demo/avl"
 11	"gno.land/p/demo/ufmt"
 12	"gno.land/r/demo/users"
 13)
 14
 15var (
 16	ErrNotFound    = errors.New("not found")
 17	StatusNotFound = "404"
 18)
 19
 20type Microblog struct {
 21	Title  string
 22	Prefix string   // i.e. r/gnoland/blog:
 23	Pages  avl.Tree // author (string) -> Page
 24}
 25
 26func NewMicroblog(title string, prefix string) (m *Microblog) {
 27	return &Microblog{
 28		Title:  title,
 29		Prefix: prefix,
 30		Pages:  avl.Tree{},
 31	}
 32}
 33
 34func (m *Microblog) GetPages() []*Page {
 35	var (
 36		pages = make([]*Page, m.Pages.Size())
 37		index = 0
 38	)
 39
 40	m.Pages.Iterate("", "", func(key string, value interface{}) bool {
 41		pages[index] = value.(*Page)
 42		index++
 43		return false
 44	})
 45
 46	sort.Sort(byLastPosted(pages))
 47
 48	return pages
 49}
 50
 51func (m *Microblog) RenderHome() string {
 52	output := ufmt.Sprintf("# %s\n\n", m.Title)
 53	output += "# pages\n\n"
 54
 55	for _, page := range m.GetPages() {
 56		if u := users.GetUserByAddress(page.Author); u != nil {
 57			output += ufmt.Sprintf("- [%s (%s)](%s%s)\n", u.Name(), page.Author.String(), m.Prefix, page.Author.String())
 58		} else {
 59			output += ufmt.Sprintf("- [%s](%s%s)\n", page.Author.String(), m.Prefix, page.Author.String())
 60		}
 61	}
 62
 63	return output
 64}
 65
 66func (m *Microblog) RenderUser(user string) string {
 67	silo, found := m.Pages.Get(user)
 68	if !found {
 69		return StatusNotFound
 70	}
 71
 72	return (silo.(*Page)).String()
 73}
 74
 75func (m *Microblog) Render(path string) string {
 76	parts := strings.Split(path, "/")
 77
 78	isHome := path == ""
 79	isUser := len(parts) == 1
 80
 81	switch {
 82	case isHome:
 83		return m.RenderHome()
 84
 85	case isUser:
 86		return m.RenderUser(parts[0])
 87	}
 88
 89	return StatusNotFound
 90}
 91
 92func (m *Microblog) NewPost(text string) error {
 93	author := std.GetOrigCaller()
 94	_, found := m.Pages.Get(author.String())
 95	if !found {
 96		// make a new page for the new author
 97		m.Pages.Set(author.String(), &Page{
 98			Author:    author,
 99			CreatedAt: time.Now(),
100		})
101	}
102
103	page, err := m.GetPage(author.String())
104	if err != nil {
105		return err
106	}
107	return page.NewPost(text)
108}
109
110func (m *Microblog) GetPage(author string) (*Page, error) {
111	silo, found := m.Pages.Get(author)
112	if !found {
113		return nil, ErrNotFound
114	}
115	return silo.(*Page), nil
116}
117
118type Page struct {
119	ID         int
120	Author     std.Address
121	CreatedAt  time.Time
122	LastPosted time.Time
123	Posts      avl.Tree // time -> Post
124}
125
126// byLastPosted implements sort.Interface for []Page based on
127// the LastPosted field.
128type byLastPosted []*Page
129
130func (a byLastPosted) Len() int           { return len(a) }
131func (a byLastPosted) Swap(i, j int)      { a[i], a[j] = a[j], a[i] }
132func (a byLastPosted) Less(i, j int) bool { return a[i].LastPosted.After(a[j].LastPosted) }
133
134func (p *Page) String() string {
135	o := ""
136	if u := users.GetUserByAddress(p.Author); u != nil {
137		o += ufmt.Sprintf("# [%s](/r/demo/users:%s)\n\n", u.Name(), u.Name())
138		o += ufmt.Sprintf("%s\n\n", u.Profile())
139	}
140	o += ufmt.Sprintf("## [%s](/r/demo/microblog:%s)\n\n", p.Author, p.Author)
141
142	o += ufmt.Sprintf("joined %s, last updated %s\n\n", p.CreatedAt.Format("2006-02-01"), p.LastPosted.Format("2006-02-01"))
143	o += "## feed\n\n"
144	for _, u := range p.GetPosts() {
145		o += u.String() + "\n\n"
146	}
147	return o
148}
149
150func (p *Page) NewPost(text string) error {
151	now := time.Now()
152	p.LastPosted = now
153	p.Posts.Set(ufmt.Sprintf("%s%d", now.Format(time.RFC3339), p.Posts.Size()), &Post{
154		ID:        p.Posts.Size(),
155		Text:      text,
156		CreatedAt: now,
157	})
158	return nil
159}
160
161func (p *Page) GetPosts() []*Post {
162	posts := make([]*Post, p.Posts.Size())
163	i := 0
164	p.Posts.ReverseIterate("", "", func(key string, value interface{}) bool {
165		postParsed := value.(*Post)
166		posts[i] = postParsed
167		i++
168		return false
169	})
170	return posts
171}
172
173// Post lists the specific update
174type Post struct {
175	ID        int
176	CreatedAt time.Time
177	Text      string
178}
179
180func (p *Post) String() string {
181	return "> " + strings.ReplaceAll(p.Text, "\n", "\n>\n>") + "\n>\n> *" + p.CreatedAt.Format(time.RFC1123) + "*"
182}

Its a bit more code, but having a full-featured microblog with users and posts and persistent data, with only a Gno standard library is pretty amazing to me.

There is a lot more information about this project that you can dive into here where you can also read aobut how to spin up your own server.

more about gno

Gno provides Go developers with a smooth transition into the world of smart contracts. With a syntax close to Go and a platform like Gno.land, developing decentralized applications becomes accessible. While Gno introduces some unique features and limitations, its potential for web3 development is promising.

• Examples galore: continue exploring with the dozens of examples of Gno.
• Getting started: more information on getting started with Gno.
• Community creations: checkout what people are building in gno.
• Knoweldge hub: read previous talks about Gno.
• Enter the conversation: join the discord.

The workshop was designed for people with all varieties of background and any amount of musical and programming experience. These Europe workshops will be taught by a combination of teachers, including myself as well as the brilliant musician Hans Teigar, along with the unbounded enthusastic creators of this workshop - Dan Derks and Jonathan Snyder.

The workshops are an incredible way to immerse yourself in an experience that aims to guide your own creativity to fulfill your musical/creative dreams, whatever they may be.

Personally I find myself constantly making norns scripts to deliver on some musical idea/dream. As one of the habitus guides this summer I will strive to relay some things I’ve learned or found helpful.

Currently there are no upcoming workshops, but stay tuned!

past events

in the past, habitus operated in these spacetimes:

• August 12 + 13, 2023 in Stockholm, Sweden (link to discussion)
• August 5 + 6, 2023 in Berlin, Germany (link to discussion)
• August 3 + 4, 2023 in Berlin, Germany (link to signup)
• September 24 + 25, 2022 @ Luck Dragon in Delhi, New York (link)
• April 1 + 2, 2023 @ CETI in Portland, Oregon (link), with the incredible Francisco Botello

This SuperCollider code also creates a graphics display of a waveform area, a playhead, and a pan symbol. The waveform area (large rectangle on each row) is drawn using a color that is dynamically adjusted based on the volume level of the audio being played. The playhead (white line) indicates the current position of the playback, and the pan symbol (small rectangle) indicates the panning position. The resulting display provides a visual representation of the audio playback and allows for visual feedback during interactive performance.

Usage / Installation

1. Go to the SuperCollider website to download and run the latest SuperCollider installer.
2. Download the latest sc3-plugins and install them into the SuperCollider extensions folder.
3. Download the latest portedplugins and install them into the SuperCollider extensions folder.
4. Download Ube.sc and also copy it into the SuperCollider extensions folder.
5. In the SuperCollider IDE you can run the following code to start it (make sure to change the file that is loaded):

(
s.waitForBoot({
	var pairs;
	// load the ube class
	u=Ube.new(Server.default);

	// load a file
	u.loadTape(tape:1,filename:thisProcess.nowExecutingPath.dirname++"/kalimba.wav");

	// play the tape with lots of players
	pairs=[
		[1, 0], // rate is the first number, db is the second
		[2, -6],
		[0.5, -6],
		[4, -9],
		[0.25, -9],
		[0.125,0],
	];
	pairs.do({ arg v,i;
		u.playTape(tape:1,player:i,rate:v[0],db:v[1],timescale:1.0);
	});

	// show gui
	u.gui;
});
)

Though I made Ube primarily as a teaching tool, its also been a fun exploratory musical instrument.

For more of my music, check out my Bandcamp.

At a certain point, Celmins selected eleven stones, and set out to make an exact copy of each one. “They seemed so beautiful that I wanted to make them myself, I wanted to see how close I could come.” Celmins believes that, if you spend enough time on a work, something else might come into play.

via Vija Celmins.

To start, you just need USB audio adapters. It might be best to get ones that have a 48k sampling rate, as thats the norns sampling rate (others should work, but it will cost CPU to resample them). You can get these as cheap as $8.

After plugging in the adapters, the norns code needs to be slightly modified to utilize the extra audio channels. First add a file, ~/.asoundrc that has bindings for each device. Here is my file for two dongles, for a total of six channels:

pcm.merge {
    type multi;
    slaves.a.pcm hw:sndrpimonome
    slaves.a.channels 2;
    slaves.b.pcm hw:1
    slaves.b.channels 2;
    slaves.c.pcm hw:2
    slaves.c.channels 2;
    bindings.0.slave a;
    bindings.0.channel 0;
    bindings.1.slave a;
    bindings.1.channel 1;
    bindings.2.slave b;
    bindings.2.channel 0;
    bindings.3.slave b;
    bindings.3.channel 1;
    bindings.4.slave c;
    bindings.4.channel 0;
    bindings.5.slave c;
    bindings.5.channel 1;
}

ctl.merge {
    type hw
    card 0
}

You can remove slave c if you only have a single dongle, or you can add more slave X if you have more than two dongles.

In this example I am using six channels, so you’ll also need to specify this in the jack systemd file. You can edit this file with sudo:

> sudo vim /etc/systemd/system/norns-jack.service

Find the line that starts with ExecStart and change it to:

ExecStart=/usr/bin/jackd -R -P 95 -d alsa -P merge -C hw:sndrpimonome -r 48000 -n 3 -p 1024 -S -s -o 6 -i 2

The -o 6 specifies 6 channels. If you have more or less you can change this number.

Finally, we have to edit the crone service to have 6 channels (or however many you have). Edit the Crone.sc file:

> vim ~/norns/sc/core/Crone.sc

And add the following line:

server.options.numOutputBusChannels = 6;

after the line that says server.latency = 0.05;. This will make sure that SuperCollider utilizes as many buses for output as you specify.

Now you can restart norns.

Finally, you have to connect jack to the outputs. To do this, you can simply run jack_connect with as many new ports added. For these examples, I have six channels - four are extra - so I can attach them using:

jack_connect system:playback_3 SuperCollider:out_3
jack_connect system:playback_4 SuperCollider:out_4
jack_connect system:playback_5 SuperCollider:out_5
jack_connect system:playback_6 SuperCollider:out_6

This will connect the new channels (3-6) to SuperCollider.

That’s it! Now you can make SuperCollider synths that output through channels 1-6 using multiexpansion. For example, here’s a diff that changes the PolyPerc engine to use six channels to output, instead of 2:

diff --git a/sc/engines/Engine_PolyPerc.sc b/sc/engines/Engine_PolyPerc.sc
index 88362aca..436de50f 100644
--- a/sc/engines/Engine_PolyPerc.sc
+++ b/sc/engines/Engine_PolyPerc.sc
@@ -17,10 +17,10 @@ Engine_PolyPerc : CroneEngine {
                pg = ParGroup.tail(context.xg);
     SynthDef("PolyPerc", {
                        arg out, freq = 440, pw=pw, amp=amp, cutoff=cutoff, gain=gain, release=release, pan=pan;
-                       var snd = Pulse.ar(freq, pw);
+                       var snd = Pulse.ar([freq,freq*2,freq/2,freq/4,freq*1.5,freq*3], pw);
                        var filt = MoogFF.ar(snd,cutoff,gain);
                        var env = Env.perc(level: amp, releaseTime: release).kr(2);
-                       Out.ar(out, Pan2.ar((filt*env), pan));
+                       Out.ar(0, (filt*env));
                }).add;

Each speaker output (or three sets of headphones) will now all have different synthesized audio!

(that’s not me).

On Linux, using multiple audio channels is quick and easy. You can buy the cheap DACs that do “USB audio” (~$10) and plug them into your computer. Then setup a ~/.asoundrc file like the following (more info about this on the Jack audio page):

pcm.merge {
    type multi;
    slaves.a.pcm hw:0
    slaves.a.channels 2;
    slaves.b.pcm hw:1
    slaves.b.channels 2;
    bindings.0.slave a;
    bindings.0.channel 0;
    bindings.1.slave b;
    bindings.1.channel 0;
    bindings.2.slave a;
    bindings.2.channel 1;
    bindings.3.slave b;
    bindings.3.channel 1;
}

ctl.merge {
    type hw
    card 0
}

This defines a new sound card “merge” that is the combination of the the first two audio devices (computer speakers and USB audio by default). You can easily add more slaves and more bindings depeneding on which cards are plugged in. The hw:0/1 are based off the cat /proc/asound/cards output. Then you can simply run JACK using this merged soundcard (named “merge”):

> jackd -v -R -P 95 -d alsa -P merge -C hw:0

(Note that the capture is still specified as hw:0).

Then in SuperCollider, you can open it and boot it with some lines specifying the number of output bus channels (in the example above we only have 4, but it could be many):

s=Server.local;
s.options.numOutputBusChannels=4;
s.boot;

Now you can use things like PanAz to pan sounds across all four channels, e.g.

(
{
	var snd;
	snd = PanAz.ar(4,SinOsc.ar(220)*0.1,MouseX.kr(-1,1),orientation:0);
	snd = snd + PanAz.ar(4,SinOsc.ar(110)*0.1,MouseX.kr(-1,1),orientation:0.25);
	snd = snd + PanAz.ar(4,SinOsc.ar(880)*0.1,MouseX.kr(-1,1),orientation:0.5);
	snd = snd + PanAz.ar(4,SinOsc.ar(442)*0.1,MouseX.kr(-1,1),orientation:0.75);
	Out.ar(0,snd.poll);
}.play;
)

More info about this in The SuperCollider Book.

First download grid from Fred’s ImageMagick Scripts: link.

It’s helpful to resize to a easy number:

convert 1.jpg -resize 1600 2.jpg

Then you can make a grid with lines in one line:

./grid -c white -s 200 -t 2 2.jpg 3.jpg

That’s it! Change 100 to 200 or 400 for bigger spacings.

I made moonraker, a norns script to generate random syncopations from samples using sequences of euclidean rhythms.

I made lorenzo’s drums, a norns script for sequencing a meticulously sampled drumset.

I made a second version of mx.samples, but found that it used slightly more CPU so I never officially released it.

I made a special version of mx.samples that only used marimba, called mallets but also never released it officially.

Speaking of unreleased scripts, there is also bb (a bouncing ball norns script), qwertymidi (a script for using a keyboard as a midi controller), io (a script that sequences using arpeggios), grdctl (like 0-ctrl but for the grid), and softdelay (a norns script mod for a simple softcut delay).

I made a norns mod to broadcast audio. This later became an entire service called stream my audio and is available on any PC.

I made acid-test, a norns script that makes generative acid basslines using markov chains. This later became oomph which used a better 303 emulation that I wrote in SuperCollider.

Speaking of SuperCollider, I wrote ouroboros, a SuperCollider script for recording seamless loops.

Speaking of seamless loops, I wrote seamlessloops, a small binary that makes loops out of any audio.

I made dnb.lua which is a Lua script that makes generative music using sox. I made a norns script based of this called makebreakbeat. This later became sampswap. This later resulted in an album based on seamless loops using a “random audio workstation”, the album is on bandcamp and all the code is open-source. There is a norns script based around all this code called paracosms which I also released.

I made a library to convert SVG images to gcode for penplotters, svg2gcode.

I made a two-channel tuner for norns called 2tuner which eventually got integrated into zxcvbn, another script that I wrote that compiles many things I’ve learned over the years into a near-complete DSP scaffold.

I scavenged code across the internet to make a easy-to-use norns that runs on a desktop, which can also be used online (currently online at play.norns.online).

I cobbled over some SuperCollider code for tape emulation into a norns script called tapedeck.

I recently made a little script that uses tapedeck with amen breaks, called amenbreak.

I made a relaxing norns script called l ll l which is like a musical emission spectrum.

I have more musical ideas to explore in 2023. Looking forward to it.

[Interface]
Address = ...
DNS = ...
PrivateKey = ...

[Peer]
Endpoint = ...
PersistentKeepalive = ...
PublicKey = ...
PresharedKey = ...
AllowedIPs = ...

Now install wireguard:

sudo apt update
sudo apt install wireguard

Now, depending on the system you need to fix resolveconf:

sudo ln -s /usr/bin/resolvectl /usr/local/bin/resolvconf

4. Now start wireguard! On the phone you just click a button. In Ubuntu you do:

sudo wg-quick up wg0

Get stats with

sudo wg show wg0

Turn it off with

sudo wg-quick down wg0

5. Enjoy ad-less browsing with a VPN.

Simply collect the stat information:

cat /proc/<pid>/stat

Add the 14th and 15th columns together and that’s the total clock ticks used by that process.

Now wait a specified duration (say 2 seconds) and collect that information again. Take the difference between total clock ticks between the two times, and then divide by the duration that elapsed. This is the total clock ticks used by that process during that time.

To convert the total clock ticks to time, you’ll need to divide clock ticks per second which you can get with a simple command:

getconf CLK_TCK

Here’s a simple Go implementation I made: example in Go and another example I found online for one written in bash: example in bash.

One proposition for how intelligent behavior may arise in an intelligent system embodied in an environment is the theory of active inference via the free energy principle (FEP) (Friston et al., 2012). The FEP suggests a testable implication that at every spatiotemporal scale, any self-organizing system separate from its environment seeks to minimize its variational free energy (VFE)

The testing from the paper:

To test whether learning reflects a reduction in VFE within BNNs, we used the information entropy of neuronal responses as a proxy for the average surprise (a.k.a. self-information), which is upper-bounded by VFE (see STAR Methods). We predicted a reduction in information entropy during the learning of gameplay. We further predicted an increase in entropy following unpredictable (random) feedback, reflecting and ensuing state of “surprise” (and, implicitly, high VFE), relative to pre-feedback states.

> sudo apt-get install imagemagick

Then run this command to convert it:

> convert input.png -colorspace gray -fill green -tint 50 tinted.png

All the code is on my github: schollz/opkit.

But basically, all I did was first trim off silence from the end of each sample using sox:

sox sample.wave new_sample.wave reverse silence 1 0.1 -50d reverse

That command will trim any silence over 100 ms at the end of the file, where silence is defined by anything at or below -50 dB.

Then I determined the pitch of each hit using aubio to calculate the most fundamental pitch:

aubio pitch -i new_sample.wav -m schmitt -u midi

That results in a list, and I used the mode to find the most common rounded midi note and then took the overage of +/-2 notes away from the mode. This worked out to be accurate in demo sounds using the “mcomb” or “schmitt” algorithms.

After that I just wrote some Golang glue that selected random files, re-pitched them to whatever note I want (say “C”), and then randomly included up to 11.5 seconds (that’s the OP limit on the sampler) and used schollz/teoperator to splice them together.

Here’s the result of a patch made from splicing kick sounds in the key of C:

The patch has extra information in the header file that tells the OP-Z and OP-1 how to calculate the sample start/end points.

And here’s a little example track I made with kick, snare, hat and bass sounds from the Pulsar-23, all pitched to “C”. Made on an OP-Z:

In the schollz/opkit there are instructions for generating your own kits, but I generated a bunch of C-based kits if you want to just try: Pulsar-23 OP-1/OP-Z kits.

Turns out this can be done in a imprecise and coarse way that does have some semblance to random binaural positioning, by using slight delays between the two channels and some filtering. Here’s a small example:

(
b = Buffer.read(s, Platform.resourceDir +/+ "sounds/a11wlk01.wav");
{
	var snd=PlayBuf.ar(1,b,loop:1);
	var pan=SinOsc.kr(1/7.13,Rand(0,6))/1.5;
	var pan2=SinOsc.kr(1/12.123,Rand(0,6))/1.5;
	var pan3=SinOsc.kr(1/15.12354,Rand(0,6))/1.5;
	var amp=SinOsc.kr(1/17.123,Rand(0,6)).range(0.25,0.75);
	snd=Pan2.ar(snd,0);
	//snd=WhiteNoise.ar(0.1)!2;
	snd=[snd[0],snd[1]];
	snd=[
		LPF.ar(snd[0],LinExp.kr((pan2<0)*pan2.abs,0,1,4000,18000).poll),
		LPF.ar(snd[1],LinExp.kr((pan2>0)*pan2.abs,0,1,4000,18000).poll)
	];
	snd[0]=SelectX.ar(((pan>0)*pan.abs),[snd[0],DelayN.ar(snd[0],0.04,0.04)]);
	snd[1]=SelectX.ar(((pan<0)*pan.abs),[snd[1],DelayN.ar(snd[1],0.04,0.04)]);
	snd=Balance2.ar(snd[0],snd[1],pan3);
	Out.ar(0,snd*amp);
}.play;
)

In this example the channels are split and then each channel undergoes some random oscilation in its low-pass filter and the side of its small delay (40 ms). There is also a traditional equal-power pan at the end. Its certainly head-swirling, although not quite at the level of true ambisonics.

This resulted in a fitting equation to get the same cruve, and luckily enough there is already a SuperCollider port of this equation from Eric Skogen. For my ears, it sounds great.

This is a great example of how a simple idea can become an amazing idea in its implementation. Detuning six oscillators is a simple thing, but the non-linear response built-in to the JP-8000 makes it way more pleasant as it increases the “sweet” spots.

Normally norns comes packaged as a standalone device or as a shield for a Raspberry Pi. However it is possible to utilize it without a physical hardware, and just run on a regular headless Raspberry Pi without the shield.

Expectations

Running norns on a headless pi without the shield misses out on several things without the physical device. Keep this in mind:

The physical device uses buttons and encoders and a screen. The system described below uses a web browser as a screen and uses your mouse to interact with the virtual buttons/encoders, which pales in comparison to the physical implementation (but works nonetheless).

The physical norns device easily interfaces with midi gear. The system described below does not (although in theory it could with Web Midi but I’m not interested in implementing that).

The physical norns device has low latency and high-quality audio. Depending on the USB audio device you get you would also have low latency and high-quality (24-bit, low noise) but it will depend on your device.

Installation on a Docker-enabled computer

Currently these instructions only work if you have an amd64 processor. If you have a Raspberry Pi, you’ll need to skip to the next instructions.

If you want to install this on a computer that runs Docker (Linux server or something), then the instructions are much simpler. You can simply clone the main directory and build the docker image:

git clone https://github.com/schollz/norns-desktop
./run_docker.sh

This will automatically start norns with an internet radio as well. If you want to use on-board audio you need to edit jackdr from the following:

/usr/bin/jackd -R -P 95 -d dummy -r 48000 -p 128

to something that uses your audio (e.g. audio device 0):

/usr/bin/jackd -R -P 95 -d alsa -d hw:0 

And then run run_docker.sh again.

Open <your pi IP address>:8889 to see your norns and interact with it. Open <your pi IP address>:5000 to use maiden to install scripts and edit things and restart the system.

Installation on a Raspberry Pi (full instructions)

Pre-requisites

• Raspberry Pi with 64-bit OS installed
• 2x2 USB interface (like this one for $10)

Install libraries

DEBIAN_FRONTEND=noninteractive sudo apt-get install -y \
  libncursesw5-dev sox git libicu-dev libudev-dev \
  libncurses5-dev libssl-dev  apt-transport-https  dbus \
  apt-utils  ca-certificates  gnupg2  build-essential  bzip2  \
  cmake  curl  gdb  ladspalist  libasound2-dev  \
  libavahi-client-dev  libavahi-compat-libdnssd-dev  \
  libcwiid-dev  libcairo2-dev  libevdev-dev  libfftw3-dev \
  libicu-dev  liblo-dev  liblua5.1-dev  liblua5.3-dev  \
  libreadline6-dev  libxt-dev  luarocks  pkg-config  \
  python-dev  unzip  wget cdbs  libboost-program-options-dev \
  make gcc g++ libmad0-dev  libid3tag0-dev libsndfile1-dev \
  libgd-dev libboost-filesystem-dev libboost-regex-dev \
  python3-pip python3-setuptools python3-wheel vim \
  libboost-dev libsdl2-dev x11vnc xvfb x11-apps imagemagick \
  icecast2 lame espeak ffmpeg vorbis-tools darkice 

Install audiowaveform

git clone https://github.com/bbc/audiowaveform.git /tmp/audiowaveform && \
    mkdir -p /tmp/audiowaveform/build && \
    cd /tmp/audiowaveform/build && \
    cmake -D ENABLE_TESTS=0 .. && make && sudo make install && \
    audiowaveform --help

Install aubio tools

git clone https://git.aubio.org/aubio/aubio /tmp/aubio && cd /tmp/aubio && \
    make && cd /tmp/aubio && sudo ./waf install --destdir=/ && sudo ldconfig && \
    cd / && rm -rf /tmp/aubio && aubioonset --help

Install Go

For 64-bit Raspberry Pi:

wget https://go.dev/dl/go1.19.2.linux-arm64.tar.gz -O /tmp/go.tar.gz && \
    sudo tar -C /usr/local -xzvf /tmp/go.tar.gz && \
    rm -r /tmp/go.tar.gz && \
    /usr/local/go/bin/go version

install ldoc

sudo luarocks install ldoc

install jack2

First install the distro-version to get the folders created.

sudo apt install jack

Then install a later version known to work v1.9.19:

export JACK2_VERSION=1.9.19 && mkdir -p /tmp/jack2 && \ 
    wget https://github.com/jackaudio/jack2/archive/v$JACK2_VERSION.tar.gz -O /tmp/jack2/jack2.tar.gz && \
    cd /tmp/jack2 && \
    tar xvfz jack2.tar.gz && \
    cd /tmp/jack2/jack2-$JACK2_VERSION && \
    ./waf configure --classic --alsa=yes --firewire=no --iio=no --portaudio=no --prefix /usr && \
    ./waf && sudo ./waf install && cd / && rm -rf /tmp/jack2 && sudo ldconfig

Install SuperCollider

export SUPERCOLLIDER_VERSION=3.12.2 &&  mkdir -p /tmp/supercollider && cd /tmp/supercollider && \
    wget https://github.com/supercollider/supercollider/releases/download/Version-$SUPERCOLLIDER_VERSION/SuperCollider-$SUPERCOLLIDER_VERSION-Source.tar.bz2 -O sc.tar.bz2 && \
    tar xvf sc.tar.bz2 && cd /tmp/supercollider/SuperCollider-$SUPERCOLLIDER_VERSION-Source && \
    mkdir -p build && cd /tmp/supercollider/SuperCollider-$SUPERCOLLIDER_VERSION-Source/build && \
    cmake -DCMAKE_BUILD_TYPE="Release" \
          -DCMAKE_INSTALL_PREFIX=/usr/local \
          -DBUILD_TESTING=OFF \
          -DENABLE_TESTSUITE=OFF \
          -DNATIVE=OFF \
          -DINSTALL_HELP=OFF \
          -DSC_IDE=OFF \
          -DSC_QT=OFF \
          -DSC_ED=OFF \
          -DSC_EL=OFF \
          -DSUPERNOVA=OFF \
          -DSC_VIM=OFF \
          .. && \
    make -j1 && sudo make install && cd /

While that is installing, you can also install the plugins:

export SUPERCOLLIDER_VERSION=3.12.2 &&  SUPERCOLLIDER_PLUGINS_VERSION=3.11.1 && \
    mkdir -p /tmp/sc3-plugins && cd /tmp/sc3-plugins && \
    git clone --depth=1 --recursive --branch Version-$SUPERCOLLIDER_PLUGINS_VERSION https://github.com/supercollider/sc3-plugins.git && \
    cd /tmp/sc3-plugins/sc3-plugins && mkdir -p build && \
    cd /tmp/sc3-plugins/sc3-plugins/build && \
    cmake -DSC_PATH=/tmp/supercollider/SuperCollider-$SUPERCOLLIDER_VERSION-Source \
          -DNATIVE=OFF \
          .. && \
    cmake --build . --config Release -- -j1 && \
    sudo cmake --build . --config Release --target install && \
    cd / && rm -rf /tmp/sc3-plugins && sudo ldconfig

Install nanomsg

export NANOMSG_VERSION=1.1.5 && mkdir -p /tmp/nanomsg && cd /tmp/nanomsg && \
    wget https://github.com/nanomsg/nanomsg/archive/$NANOMSG_VERSION.tar.gz -O nanomsg.tar.gz && \
    tar -xvzf nanomsg.tar.gz && cd /tmp/nanomsg/nanomsg-$NANOMSG_VERSION && \
    mkdir -p /tmp/nanomsg/nanomsg-$NANOMSG_VERSION/build && \
    cd /tmp/nanomsg/nanomsg-$NANOMSG_VERSION/build && \
    cmake .. && cmake --build . && sudo cmake --build . --target install && \
    cd / && rm -rf /tmp/nanomsg && sudo ldconfig

Install libmonome

export LIBMONOME_VERSION=1.4.4 && cd /tmp/ && wget https://github.com/monome/libmonome/archive/v$LIBMONOME_VERSION.tar.gz -O libmonome.tar.gz && \
    tar -xvzf libmonome.tar.gz && cd /tmp/libmonome-$LIBMONOME_VERSION && \
    ./waf configure --disable-udev --disable-osc && \
    ./waf && sudo ./waf install && \
    cd / && rm -rf /tmp/libmonome-$LIBMONOME_VERSION && sudo ldconfig

Create credentials

Edit the /etc/security/limits.conf file and add these two lines:

@audio   -  rtprio     95
@audio   -  memlock    unlimited

Now lets create the we user:

sudo useradd -g audio we -m -s /bin/bash
sudo usermod -aG video we
sudo adduser we sudo 
sudo mkhomedir_helper we

Make sure to add we to the sudoers:

sudo visudo
# add
we ALL=(ALL) NOPASSWD: ALL

Add a password for we:

sudo passwd we 

Very important now to log in as we:

su we
cd ~

Install oh-my-zsh

Now you should always be logged in as we!

This makes things easier.

ZSH=/home/we/.oh-my-zsh sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

curl https://raw.githubusercontent.com/schollz/dotfiles/master/.zshrc > ~/.zshrc
source ~/.zshrc

Install node

(You should be logged in as we).

For 64-bit Raspberry pi:

cd /tmp/ && wget https://nodejs.org/dist/v16.17.1/node-v16.17.1-linux-arm64.tar.xz -O /tmp/node.tar.xz && \
  mkdir -p /home/we/node && tar -xJf /tmp/node.tar.xz -C /home/we/node && \
  mv /home/we/node/node-*/* /home/we/node/ && \
  mkdir -p /home/we/node/node_modules && \
  npm config set prefix "/home/we/node/node_modules" && \
  npm install -g npm yarn && npm -v && node -v

Install maiden

(You should be logged in as we).

This step uses a specially patched version of maiden that doesn’t use dbus.

export MAIDEN_TAG=ce4471e25a45c87040817c0619f3596fa43060aa &&
    export MAIDEN_REPO=https://github.com/schollz/maiden.git && git clone $MAIDEN_REPO maiden_src && \
     cd maiden_src && \
     git checkout $MAIDEN_TAG && \
     make release-local && \
     tar -xvf dist/maiden.tgz -C /home/we && \
     /home/we/maiden/project-setup.sh

Install matron

(You should be logged in as we).

At this step you might need a little more than 1G of ram to compile. Create a small swpa in case you haven’t to make sure it compiles fine:

sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Now you can build norns:

export NORNS_TAG=2faa96bd763b69e4b840adc4e2ee8e58e00522f0 && \
  export NORNS_REPO=https://github.com/monome/norns.git && \
  cd /home/we/ && git clone $NORNS_REPO &&  cd /home/we/norns && \
  git checkout $NORNS_TAG &&  git submodule update --init --recursive && \
  ./waf configure --desktop && ./waf build --desktop

Once installed, the norns source needs to be patched:

sed -i 's/norns.disk/100000/g' /home/we/norns/lua/core/menu/tape.lua
sed -i 's/screensaver.time = 900/screensaver.time = 90000000/g' /home/we/norns/lua/core/screen.lua

Also you need to a copy a file to the SuperCollider extensions:

mkdir -p ~/.local/share/SuperCollider/Extensions
cp /home/we/norns/sc/norns-config.sc ~/.local/share/SuperCollider/Extensions/

Install startup files

(You should be logged in as we).

cd /home/we && git clone https://github.com/schollz/norns-desktop && \
    cd /home/we/norns-desktop && chmod +x *sh && \
    cp restart_* /home/we/norns/ && \
    cp /home/we/norns-desktop/matronrc.lua /home/we/norns/ && \
    go build -v -x

Setup the main directories and add some scripts:

mkdir -p ~/dust/data && mkdir -p ~/dust/audio && \
    mkdir -p ~/dust/code && cd /home/we/dust/code && \
    git clone https://github.com/tehn/awake && \
    git clone https://github.com/northern-information/dronecaster && \
    git clone https://github.com/schollz/l_ll__l_

Run norns!

(You should be logged in as we).

You can run norns in the full-mode (with playback and recording) only if you have a 2x2 USB audio interface. Otherwise you need to run in playback mode only.

By default the script is for playback mode, only:

/home/we/norns-desktop/start_norns_pi.sh

If you want to use a 2x2 USB audio interface, you need to change start_norns_pi.sh and uncomment the line that says 2x2 USB interface and comment the line that says playback only.

Open <your pi IP address>:8889 to see your norns and interact with it. Open <your pi IP address>:5000 to use maiden to install scripts and edit things and restart the system.

Troubleshooting

Each song combines over a dozen samples, each from a different artist. Each song is a paracosm: a random imagined world emerging from a combination of creators.

Though there are many songs, they all follow a similar theme which I would vaguely describe as glitched ambient jungle usually with instrumentation from synths, strings, pianos and saxophones. The album can be listened in order from track 1 to track 100 - each song changes key from one to next according to the circle of fifths to promote a sense of progression. Or listen in any order you want.

the process

My goal for this album was to utilize a process that used only pre-recorded samples. I usually avoid this type of process, for various reasons. (Mostly I like performance-based music akin to my musical-origins of playing piano). But my motivation for this project was to immerse myself in my own personal reluctance.

To start, I needed samples. I curated a catalog of pre-recorded samples by randomly selecting things I liked from splice.com. I spent $110 to collect all the samples I needed to make the entire album.

The process of manipulating samples into a song can be laborious - it requires taking a sample in a different key or tempo and stretching it to the target tempo, re-pitching it, trimming it to the right length, adding the right effects, splicing it in with crossfades in the right way, etc. For a hundred song album, this would mean doing these microedits thousands of times.

To avoid the tedium in using samples, I wrote a program I call raw (github.com/schollz/raw). The “daw” is familiar as the “digital audio workstation” but this audio workstation made almost all decisions stochastically so I call it the random audio workstation" (e.g. raw). raw randomly chooses which samples to use, how to layer the samples, how to do transitions, and which effects to add. The decisions are all based around probabilities so the same set of samples might create an entirely different song. I have control over the probabilities but not much else.

I didn’t write the entire program from scratch - raw was built upon freely available open-source music tools - sox (the music swiss army knife) and SuperCollider (a powerful music coding language). I used sox to perform all the splicing / stretching operations / effects, and I used SuperCollider for additional special effects like special gated delays and filter ramps.

(raw is a more sophisticated version of a norns script I wrote called sampswap which itself is a more sophisticated version of a norns script I wrote called makebreakbeat).

more process

I realize that my process of pooling random samples and juxtaposing them randomly is a bit alternative so I thought I’d describe in detail pieces of it.

I have all of my downloaded samples in a single directory. To make matters easier I renamed each file to include its tempo. I auditioned songs by listening to them back in a web browser that pointed to the directory of samples. This little website I made that has code that is so poorly written I don’t bother to share it.

The screenshot of the website for sample auditioning is above. Its a very basic UI that I can scroll through my samples in parallel and play them back to see which pool of samples might work well together. Its really dumb simple - written in Vue and it generates a bash script that will copy the necessary files into a new directory so that they can be utilized by raw.

After that I simply run raw which generates 4-5 stems: one for drums, bass, chords, melody and sometimes vocals. This process takes 1-2 minutes to generate everything. The end result are 4-5 music files, each is a single stem for each part (drums, bass, chords, etc.), and each is the length fo the final song.

The last thing to do is to take these stems and mix them together in a way that makes sense. I usually prefer to make the drum sounds louder and vocals louder in the mix. To do this I just open them in Ableton and edit the mixing levels.

In Ableton I don’t do much - the stems each contain the entirety of one instrument of the song. After that I listen through, fix any pops that occurred from the splicing program, and render it!

Each song took between 15-30 minutes to go through the entire process. I probably generated around 300 songs for this album (rejecting >60% of them) so it was easily about 150 hours of work. If I wasn’t automating most of the steps I imagine it would’ve taken much much longer.

the art in the art

during this process I was thinking - what is the art in this process? I had no part in creating the samples - they are all pre-recorded from other musicians who sent them to splice.com. and though I wrote software to make the songs, the software itself works through randomness and makes the main choices (which samples to layer, which effects, when) without my input. So where’s the artist input?

late into the process I found that I could maintain a sense of “artistic direction” by harnessing three things within my control:

• curating of the pool of samples.
• choosing of the tempo.
• mixing the final five stems.

They seem fairly subtle, but actually in my process they had a big impact.

The software I wrote will layer samples from the pool randomly, but the pool of samples is created by me. it took a few dozen songs to get the hang of this, but I found I would need to select a pool of samples that can intermingle in a variety of ways, otherwise it can end up sounding like multiple radio stations playing simultaneously.

Also I found that my chosen tempo made a big difference - sometimes tempos were fast and sounded like a mishmash of radio stations and sometimes they were so slow they just plodded. I spent a lot of time just trying to figure out the right tempo for each track. Usually I generated a song two or three times to get one that I liked.

Another realization I had was that this whole process is basically a lesson in mixing. my software didn’t mix them - it just outputs things at the levels they were prerecorded. I could get better results by working to mix them (really just manipulating five numbers, but boy those numbers make a big difference).

enjoy

In the end I enjoyed the process (though only because it was automated) and I am enjoying the resulting album. This album has been my go-to listening for months and I’ve listened all the way through almost a dozen times. maybe you will enjoy it too. feel free to pwyw or just download it with a code:

The codes can be redeemed at https://infinitedigits.bandcamp.com/yum .

btw, the cover art

The artwork for the album utilizes Disco Diffusion, an AI that can create artworks from text. The text I chose is artwork of Savlador’s Dali’s terrariums. Dali wrote extensively about the importance of keeping spider terrariums which were like little paracosms meant to deliver inspiration.

“Oh, two days! The labour of two days, then, is that for which you ask two hundred guineas!”

“No;—I ask it for the knowledge of a lifetime.”

This quote about charging high prices for small artworks eventually got attributed to Picasso apocryphally. Picasso’s paintings and drawings are among the most valuable, though he has a great many drawings that are exceedingly simplistic. In fact, Picasso create a whole series of drawings using sometimes just a single-line.

These drawings are simple and beautiful. I got to thinking about reproducing them. But instead of printing them with an inkjet printer, I was interested in drawing them in the style they were meant to be created - using one line.

Converting a drawing to a one-line drawing

The drawings I am interested in reconstructing are done in one (but sometimes two or three) lines. To figure out the line, I would need to do several steps. I will be starting from a plain image - like a .png or .jpg file. The .png/.jpg is called a raster image. That is - it has information about pixels and intensities but no information about lines.

I would need a program to trace out the lines in the image and convert it to a coordinate system describing lines. This is what vector graphics are for - and there is a image format already encompassing it called scalable vector graphics (SVG).

Once I have the SVG file with the lines, I would need to gather up all the coordinates and the lines and consolidate the lines so that it creates a single seamless line.

Converting a raster image to vector graphics

I am going to start with a simple line drawing - this one-line drawing of a person.

Its pretty clear to see the one line with our human eyes, the challenge here will be to get a computer to also recognize it. The first thing to do is to convert it to a SVG, and we can use two programs: imagemagick and autotrace.

First, using imagemagick’s convert command, the image is thresholded and output into a format nessecary for the next tool.

> convert person.jpg -threshold 60% person.tga

Next the resulting image person.tga can be converted using autotrace into a vector graphic file using the “centerline” algorithm that finds the skeleton center line.

> autotrace -output-file person.svg --output-format svg --centerline person.tga

Great, now we have an SVG file (person.svg) to work with for the next step.

Parsing SVG and consolidating lines

Parsing a SVG is standard thing to do and easy to do with lots of tools. The actual SVG content contains a list of coordinates in “paths” which are denoted by “line commands”. Some of the coordinates are Bezier curves definitions which themselves need to be converted to absolute coordinates.

Also an SVG that looks like it contains only one line may not in fact contain one line. For example, part of the person.svg path looks like:

1...d="M47 31L45 33L47 31M127 83L139.821...
2      ^                 ^                  

There are special letters (like “M”) denoted where the path should move before starting the next stroke. The presence of these moves means that the SVG I have is actually multiple lines and not a single line.

I need to join these lines together, but first I need to re-order them in the correct order. To re-order the lines I wrote a simple algorithm that uses gradient descent to minimize the distances between subsequent end and start points of each line. This randomly looks through all possible combinations of lines until it finds the best one (though its not guaranteed to be the best).

Once I have that I can output a little animation showing the way the line moves by following the coordinates of my freshly ordered lines:

Now that I have coordinates I can get to the drawing.

Now for the actual drawing

This year my goal is to learn CAD design and to facilitate that I invested in a 3D printer. Of course 3D printers use plastic to make 3D things, but ironically one of my first 3D prints was to make a pen holder so I could make 2D things - I printed out a compatible pen holder (Thingiverse thing:4764350 designed by Pete Manville).

Now to convert the SVG to Gcode. Gcode essentially produces an instruction to move in the X/Y/Z axis. Using special instructions ("G90" and “G91”) you can also switch between absolute/relative coordinate systems. can communicate in absolute and relative coordinates. I can just parse my SVG coordinates, and for each new line it will switch to relative coordinates, move the pen up (in the Z-axis), switch to absolute coordinates, move to the new line start, switch to relative coordinates, then move back down, and finally switch to absolute coordinates and go ahead and draw. It looks something like this:

G91 ; Set coordinates to relative
G1 Z10 F1000 ; raise pen
G90 ; Set coordinates to absolute
G1 X42.712 Y51.772 F1000
G91 ; Set coordinates to relative
G1 Z0 F1000 ; lower pen
G90 ; Set coordinates to absolute
G1 X41.137 Y52.487 F1000

Since 3D/CNC machines talk to computers over Serial, its simple enough for my program to simply upload the Gcode after converting the file so that it can draw.

And that’s it!

As a bonus I can use this same code to draw my own SVG’s and upload them to print really easily with this workflow.

Source code

The source code for everything in this post can be found on my Github: schollz/svg2gcode. The basic workflow is as follows:

> # imagemagick to threshold image
> convert IMAGE -resize 300x -background White -gravity center -threshold 60% 1.tga
> # autotrace to get center line 
> autotrace -output-file potrace.svg --output-format svg --centerline 1.tga
> # convert to Gcode
> svg2gcode convert --debug --png --in potrace.svg --out IMAGE -x 0 -y 0 --width 90 --height 90 --simplify 0.005 --consolidate 0.07 --min-length 0.03 --animate
> # upload to printer
> svg2gcode upload IMAGE.gcode

I found Deep Music Visualizer, that uses the BigGAN neural network, which is great for generating tempo and pitch synced music videos.

Here’s some example videos with some short snippets of music I generated:

Its actually easy to make this style of video and it can be done for free! You also don’t need to download any software to do it - it can all be done in the browser. Here’s the tutorial on how to make your own deep AI music videos with your own music.

7 steps to make deep dream music videos

1. Get a audio file

This will probably work best with music, but any audio file will do. Make sure it is a mp3 though, and rename the audio to “1.mp3”. This is important.

(Its possible to use files with different names but if you have a different name you have to change some things further down but its more complicated).

2. Download a colab notebook

Now use this link to download the “notebook”: deep_dreams_music_video.ipynb. Click the “Download ZIP” button on that page to download it. Unzip the archive and note where it is on the computer. We will use the file in the next step.

3. Setup Google Colab

We are going to use Google’s “Colab notebooks” to generate the videos. This will allow you to generate the videos without having to install anything (and the video generation is fast because you can use Google’s GPU). And its free.

You have to have a Google account for this. Once you do, goto colab.research.google.com. Once you are on that page you might be greeted by a box such as below - if not in the upper left-hand corner you can click “File” and then “Open”. Then you should see this box:

You’ll need to click the tab that says “Upload”. Now you go ahead and hit “Browse…” and select the file you downloaded and unzipped in the last step. Now you should have it loaded in your window!

4. Change runtime

The default runtime processor is CPU, we need to use GPUs. Click “Runtime” and click “Runtime Type” and then select “GPU” where it says “Hardware accelerator”. Then hit “Save”.

5. Load the notebook

Now we can load the notebook. You can run each section of the notebook, one at a time, in sequence. Start with the first two sections.

6. Upload your music

Now we should be at the third “cell” of the notebook. Run it like you did in the last two steps. After you run it, a little button will appear that you can use to upload your music.

This button says “Browse” and you can click on it and upload your music file “1.mp3”. Wait a few seconds for it to upload.

7. Generate a music video!

Now the final part is to run the last cell of the notebook by clicking play. It will use the music file you uploaded to generate a random music video. Once its down it will be downloaded to your computer. This might take a few minutes, depending on the size of the file.

You can do this step as many times as you want - it will generate a new random video each time. You don’t need to run all the other cells if you want to make a new one either! Just run this last cell and wait a few minutes for the new video to download to your computer.

8. (optional) Modify the kinds of images

You can change the types of images shown, to an extent. This is totally optional - by default the classes of images will be random. But you can edit one line of code to try to steer the randomness by allowing only certain classes of images.

First, check out this list of available image classes. Each one has a number, and you need to write down the numbers of the classes you want. For example, if I wanted “hen” and “vulture” type images I would write down “8” and “23”.

Then change one line of code in the last cell. Instead of

1cls1000 = list(range(1000))

change it to this:

1cls1000 = [8,23]*12

I used “8” and “23” because I wanted hens and vulture type classes. But you could do any numbers you want, and choose between 2 and 12 numbers. Keep the *12 there otherwise it won’t have enough classes to compute. Try generating with that and you should get images that revolve mostly around those selected classes.

Enjoy

If you have any questions, feel free to reach out to me through Instagram or through my website. If you enjoy my music, check out the rest of my music on Bandcamp. If you enjoy my coding projects, consider becoming a sponsor.

This program runs on Windows, Mac OS, and Linux and will directly stream from any audio interface on your computer to a website I setup called streammyaudio.com.

You can get started by downloading a release for your computer and double-clicking it. The software automatically detects your audio devices and creates high quality mp3 streams (up to 260 kpbs) and forwards them the streaming server.

Anti-social media

Streaming high-quality audio from your computer to the internet shouldn’t be hard but it doesn’t seem to be easy.

Pretty much every social media site lets you share audio and video streams (e.g. Twitch, Clubhouse, YouTube, Facebook, Instagram, Discord). But each of these social media sites are now walled gardens that prevent non-subscribers from accessing content and to force people to buy into their platform (sometimes literally). What if I just want to share something without making an account?

Also the social media audio streaming is often poor quality. These social media sites sacrifice the audio quality to improve the video quality and lack the ability to modify the settings. What if I want to forego video and just have really high quality audio?

I’m not aware of a simple audio streaming solution that just works, and doesn’t require making accounts and can deliver good audio quality to anyone with a link on the internet.

Open-source audio streaming is here

It turns out we have high-quality free technology at our fingertips: ffmpeg. Normally ffmpeg is used for videos, but is great for converting between audio codecs and is also a great cross-platform way of capturing audio from any audio device. The only other thing you need then is a way to send the mp3 data to a server. And we can use another ubiquitous tool for that: curl. Although, in the final version of the streaming software I just rewrote the needed part of curl.

As I mentioned before in my blog, then all you need to send out mp3 data from your microphone is a simple line of code:

1>ffmpeg -f alsa -i hw:0 -f mp3 - | \
2    curl -s -k -H "Transfer-Encoding: chunked" -X POST -T - \
3    "https://streamyouraudio.com/YOURSTATIONNAME.mp3?stream=true&advertise=true"

But not everyone has a linux computer nor wants to know about lines of code they have to type. A better solution would be one that you just click and run, and that’s what I’ve released.

I use it to share streams of my piano performances, or sometimes just stream some of my music so I can listen to it elsewhere. Whatever the case it works great, is easy, and cheap. The server running all the code costs only a few dollars every year (if you’d like to support the cost, consider sponsoring me).

In any case, I never have to Google “how do I stream my audio” again.

The heavy-lifting is done by sox but I employed Lua to create pure functions that can perform the sox commands on audio files. For example, a function to trim the silence from both ends of an audio file is declared as:

1function audio.silence_trim(fname)
2  local fname2=string.random_filename()
3  os.cmd("sox "..fname.." "..fname2.." silence 1 0.1 0.025% reverse silence 1 0.1 0.025% reverse")
4  return fname2
5end

My idea to produce “generative” breakbeat music was to take a loop and perform operations randomly, in sequence, so that the loop itself has memory of each operation and thus the result can be more complex than utilizing the original file for each.

When copying and pasting audio its important to be aware of the boundaries. Often the boundaries are “merged” by crossfading some “excess” region on the outside of the regions you are pasting. For example, from the sox docs:

      length1   excess
    -----------><--->
    _________   :   :  _________________
             \  :   : :\     
              \ :   : : \     
               \:   : :  \     
                *   : :   *  
                 \  : :   :\    
                  \ : :   : \     
    _______________\: :   :  \_________
                      :   :   
                      <--->   
                      excess  

Its a bit like cutting and pasting two tape loops together. Practically speaking, this means that when copying a loop you actually need to copy a little bit extra for both sides and paste it at a position slightly before, so that it gets into the correct location and correctly crossfades. This bit is probably the most complicated operation as it requires multiple sox commands.

 1function audio.copy_and_paste(fname,copy_start,copy_stop,paste_start,crossfade)
 2	local copy_length=copy_stop-copy_start
 3	if copy_length==nil or copy_length<0.05 then 
 4		do return fname end 
 5	end
 6	local piece=string.random_filename()
 7	local part1=string.random_filename()
 8	local part2=string.random_filename()
 9	local fname2=string.random_filename()
10	local splice1=string.random_filename()
11	local e=crossfade or 0.1 
12	local l=0 -- no leeway
13	os.cmd(string.format("sox %s %s trim %f %f",fname,piece,copy_start-e,copy_length+2*e))
14	os.cmd(string.format("sox %s %s trim 0 %f",fname,part1,paste_start+e))
15	os.cmd(string.format("sox %s %s trim %f",fname,part2,paste_start+copy_length-e))
16	os.cmd(string.format("sox %s %s %s splice %f,%f,%f",part1,piece,splice1,paste_start+e,e,l))
17	os.cmd(string.format("sox %s %s %s splice %f,%f,%f",splice1,part2,fname2,paste_start+copy_length+e,e,l))
18	os.cmd(string.format("rm -f %s %s %s %s",piece,part1,part2,splice1))
19	return fname2
20end

(^ clicking that will expand the rest of the functions if you’d like to see them.)

Lets get to the music

I’ll go through some of the pure functions I’ve used for creating generative breakbeat music. First, here is the original audio loop that I will demonstrate with:

The original audio is great, but its short and can get repetitive so using a bunch of operations on it we can add repeats with variety.

Jumping

“Jumping” is what I call when you simply copy and paste one region to another region within the sample.

The code for “jumping” is that audio.copy_and_paste function that I posted previously. Running that function a few times on the original sample yields something quite interesting.

Reversing

Reversing is what I call when you take a region and reverse it. Reversing an audio file is one of the simplest things and it can sound great when pasted back into the original audio.

The reversing function is really simple.

1function audio.reverse(fname)
2  local fname2=string.random_filename()
3  os.cmd(string.format("sox %s %s reverse",fname,fname2))
4  return fname2
5end

But pasting it requires taking notice of the crossfade regions. I altered the audio.copy_and_paste function to allow pasting any piece of audio with crossfades. Without the crossfading it will inevitably produce “clipping” or “popping” sounds.

 1function audio.paste(fname,piece,paste_start,crossfade)
 2	local copy_length=audio.length(piece)
 3  if copy_length==nil then 
 4    do return fname end 
 5  end
 6	local part1=string.random_filename()
 7	local part2=string.random_filename()
 8	local fname2=string.random_filename()
 9	local splice1=string.random_filename()
10	local e=crossfade or 0.1 
11	local l=0 -- no leeway
12	os.cmd(string.format("sox %s %s trim 0 %f",fname,part1,paste_start+e))
13	os.cmd(string.format("sox %s %s trim %f",fname,part2,paste_start+copy_length-e*3))
14	os.cmd(string.format("sox %s %s %s splice %f,%f,%f",part1,piece,splice1,paste_start+e,e,l))
15	os.cmd(string.format("sox %s %s %s splice %f,%f,%f",splice1,part2,fname2,paste_start+copy_length+e,e,l))
16  os.cmd(string.format("rm -f %s %s %s",part1,part2,splice1))
17	return fname2
18end

Combining the audio.reverse and audio.paste on the original file creates the following:

Stutter

The stutter is my favorite effect. It is where you clip a piece of audio and then paste it many times at 1/16th note apart. At each new piece its fun to increase the volume or open up a filter.

 1function audio.stutter(fname,stutter_length,pos_start,count,crossfade_piece,crossfade_stutter,gain_amt)
 2  crossfade_piece=0.1 or crossfade_piece
 3  crossfade_stutter=0.005 or crossfade_stutter
 4  local partFirst=string.random_filename()
 5  local partMiddle=string.random_filename()
 6  local partLast=string.random_filename()
 7  os.cmd(string.format("sox %s %s trim %f %f",fname,partFirst,pos_start-crossfade_piece,stutter_length+crossfade_piece+crossfade_stutter))
 8  os.cmd(string.format("sox %s %s trim %f %f",fname,partMiddle,pos_start-crossfade_stutter,stutter_length+crossfade_stutter+crossfade_stutter))
 9  os.cmd(string.format("sox %s %s trim %f %f",fname,partLast,pos_start-crossfade_stutter,stutter_length+crossfade_piece+crossfade_stutter))
10  gain_amt=gain_amt or (count>8 and -1.5 or -2)
11  for i=1,count do 
12    local fnameNext=""
13    if i==1 then 
14      fnameNext=audio.gain(partFirst,gain_amt*(count-i))
15    else
16      fnameNext=string.random_filename()
17    local fnameMid=i<count and partMiddle or partLast 
18    if gain_amt~=0 then 
19      fnameMid=audio.gain(fnameMid,gain_amt*(count-i))
20    end
21      os.cmd(string.format("sox %s %s %s splice %f,%f,0",fname2,fnameMid,fnameNext,audio.length(fname2),crossfade_stutter))
22    end
23    fname2=fnameNext
24  end
25  return fname2
26end

This effect is also the most complicated because each little piece is crossfaded with each other little piece, but then all the pieces together must be pasted in with crossfades on either side. So I wrote the function to encompass the left, middle, and right sides so you can have shorter crossfades in the middle and longer crossfades to paste it into the original audio.

Reverse reverb

Reversing a reverb is another one of my favorite effects. This is one that is useful to have resampling (all these other effects could be done in realtime using a good sample player).

Reversing a reverb is pretty much how it sounds - take a slice and add reverb. Render the reverb ringing out and then reverse the whole thing.

While sox has a reverb built-in as well, I decided to use SuperCollider instead. SuperCollider can actually be run in “non-realtime” mode in which case you can use the SuperCollider toolkit and immediately (and pretty quickly) render audio with any of its building blocks.

(
var oscScore;
var mainServer;
var nrtServer;
var serverOptions;
var scoreFn;

mainServer = Server(\sampswap_nrt, NetAddr("127.0.0.1", 47112));
serverOptions=ServerOptions.new.numOutputBusChannels_(2);
serverOptions.sampleRate=48000;
nrtServer = Server(\nrt, NetAddr("127.0.0.1", 47114), options:serverOptions);
SynthDef("lpf_rampup", {
    arg out=0,  dur=30, f1,f2,f3,f4;
    var duration=BufDur.ir(0);
    var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
    snd=LPF.ar(snd,XLine.kr(200,20000,duration));
    snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.005,dur-0.01,0.005]), doneAction:2);
    Out.ar(out, snd);
}).load(nrtServer);
SynthDef("lpf_rampdown", {
    arg out=0,  dur=30, f1,f2,f3,f4;
    var duration=BufDur.ir(0);
    var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
    snd=LPF.ar(snd,XLine.kr(20000,200,duration));
    snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.005,dur-0.01,0.005]), doneAction:2);
    Out.ar(out, snd);
}).load(nrtServer);
SynthDef("dec_ramp", {
    arg out=0,  dur=30, f1,f2,f3,f4;
    var duration=BufDur.ir(0);
    var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
    snd=SelectX.ar(Line.kr(0,1,duration/4),[snd,Decimator.ar(snd,8000,8)]);
    snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.005,dur-0.01,0.005]), doneAction:2);
    Out.ar(out, snd);
}).load(nrtServer);
SynthDef("dec", {
    arg out=0,  dur=30, f1,f2,f3,f4;
    var duration=BufDur.ir(0);
    var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
    snd=Decimator.ar(snd,8000,8);
    snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.005,dur-0.01,0.005]), doneAction:2);
    Out.ar(out, snd);
}).load(nrtServer);
SynthDef("reverberate", {
    arg out=0,  dur=30, f1,f2,f3,f4;
    var duration=BufDur.ir(0);
    var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
    snd=SelectX.ar(XLine.kr(0,1,duration/4),[snd,Greyhole.ar(snd* EnvGen.ar(Env.new([0, 1, 1, 0], [0.1,dur-0.2,0.1]), doneAction:2))]);
    snd=LeakDC.ar(snd);
    snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.1,dur-0.2,0.1]), doneAction:2);
    Out.ar(out, snd);
}).load(nrtServer);
SynthDef("filter_in_out", {
    arg out=0,  dur=30, f1,f2,f3,f4;
    var duration=BufDur.ir(0);
    var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
    snd = RLPF.ar(snd,
        LinExp.kr(EnvGen.kr(Env.new([0.1, 1, 1, 0.1], [f1,dur-f1-f2,f2])),0.1,1,100,20000),
        0.6);
    snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.005,dur-0.01,0.005]), doneAction:2);
    Out.ar(out, snd);
}).load(nrtServer);
SynthDef("tapedeck", {
	arg out=0,  dur=30,f1,f2,f3,f4,
	amp=0.9,tape_wet=0.95,tape_bias=0.9,saturation=0.9,drive=0.9,
	tape_oversample=1,mode=0,
	dist_wet=0.07,drivegain=0.5,dist_bias=0.5,lowgain=0.1,highgain=0.1,
	shelvingfreq=600,dist_oversample=1,
	hpf=60,hpfqr=0.6,
	lpf=18000,lpfqr=0.6;
	var duration=BufDur.ir(0);
	var snd = PlayBuf.ar(2,0,BufRateScale.kr(0));
	snd=snd*amp;
	snd=SelectX.ar(Lag.kr(tape_wet,1),[snd,AnalogTape.ar(snd,tape_bias,saturation,drive,tape_oversample,mode)]);	
	snd=SelectX.ar(Lag.kr(dist_wet/10,1),[snd,AnalogVintageDistortion.ar(snd,drivegain,dist_bias,lowgain,highgain,shelvingfreq,dist_oversample)]);				
	snd=RHPF.ar(snd,hpf,hpfqr);
	snd=RLPF.ar(snd,lpf,lpfqr);
	snd = snd * EnvGen.ar(Env.new([0, 1, 1, 0], [0.005,dur-0.01,0.005]), doneAction:2);
	Out.ar(out, snd);
}).load(nrtServer);

scoreFn={
    arg inFile,outFile,synthDefinition,durationScaling,oscCallbackPort,f1,f2,f3,f4;
    Buffer.read(mainServer,inFile,action:{
        arg buf;
        Routine {
            var buffer;
            var score;
            var duration=buf.duration*durationScaling;

            "defining score".postln;
            score = [
                [0.0, ['/s_new', synthDefinition, 1000, 0, 0, \dur,duration,\f1,f1,\f2,f2,\f3,f3,\f4,f4]],
                [0.0, ['/b_allocRead', 0, inFile]],
                [duration, [\c_set, 0, 0]] // dummy to end
            ];

            "recording score".postln;
            Score(score).recordNRT(
                outputFilePath: outFile,
                sampleRate: 48000,
                headerFormat: "wav",
                sampleFormat: "int24",
                options: nrtServer.options,
                duration: duration,
                action: {
                    Routine {
                        postln("done rendering: " ++ outFile);
                        0.2.wait;
                        NetAddr.new("localhost",oscCallbackPort).sendMsg("/quit");
                    }.play;
                }
            );
        }.play;
    });
};
mainServer.waitForBoot({
    Routine {
        "registring osc for score".postln;
        oscScore = OSCFunc({ arg msg, time, addr, recvPort;
            var inFile=msg[1].asString;
            var outFile=msg[2].asString;
            var synthDefinition=msg[3].asSymbol;
            var durationScaling=msg[4].asFloat;
            var oscCallbackPort=msg[5].asInteger;
            var f1=msg[6].asFloat;
            var f2=msg[7].asFloat;
            var f3=msg[8].asFloat;
            var f4=msg[9].asFloat;
            [msg, time, addr, recvPort].postln;
            scoreFn.value(inFile,outFile,synthDefinition,durationScaling,oscCallbackPort,f1,f2,f3,f4);
            "finished".postln;
        }, '/score',recvPort:47113);
        1.wait;
        "writing ready file".postln;
        File.new("/tmp/nrt-scready", "w");
        "ready".postln;
    }.play;
});
)

I can still write a pure function to render the audio file, but I’ll be using OSC to communicate with the NRT SuperCollider server to produce the result.

 1function audio.supercollider_effect(fname,effect,f1,f2,f3,f4)
 2  local fname2=string.random_filename()
 3  local durationScaling=1 
 4  if effect=="reverberate" then 
 5    durationScaling=4
 6  end
 7  f1=f1 or 0
 8  f2=f2 or 0
 9  f3=f3 or 0
10  f4=f4 or 0
11  os.cmd(string.format(SENDOSC..' --host 127.0.0.1 --addr "/score" --port 47113 --recv-port 47888 -s %s -s %s -s %s -s %s -s 47888 -s %f -s %f -s %f -s %f',fname,fname2,effect,durationScaling,f1,f2,f3,f4))
12  return fname2
13end

Filter in/out

Speaking of SuperCollider effects…since I’m already using the NRT server I opted to add some other fancy effects that sox can’t quite do. For example - adding a filter to the beginning opening and a closing filter at the end.

Its quite easy to add these effects using the same function above, just specifying which effect it is.

All together now

Each of those effects works by itself, but then their combination can be quite cool. By adding each effect with a given probability, at a random position you can quickly get a beat with variety and lots of movement.

Usage

You can find the sources for all these files here: https://github.com/schollz/sampswap

There is also more usage information there.

Generative music

Generative or “automatic” music is basically meaning that either composition or performance (or both) is created through some algorithm, recipe, or program. There are many approaches to generative or “automatic” music composition. Approaches range from deep-learning interpretations of music, to clever mapping of musical space to physical coordinates, to musical emergence from mathematical formulas, to developing frameworks of musical structure (composition through decomposition).

I won’t go through them all the possibilities here, suffice to say that there are lot and not all of them sound good (to me! sound is of course subjective, so all this is based on my own preference). To me, the deep-learning music examples are the most artifical sounding and I’d prefer not to listen to them. To me, the methods that sound best are the ones that seek heuristics based on particular musical aesthetics (like the latter example that reconstructs classical music from core paradigms).

How to make music from a toy synthesizer

I first started thinking about generative music when I was trying to make music from a cheap toy synthesizer. I had bought a Korg Monotron Delay and I wanted to make a good sounding album from it but was challenged because it only outputs one note at a time. I decided to make a looper so I could loop notes atop each other and then tried to figure out methods for layering the notes in interesting ways. After hacking the toy synth I created a little looper that would loop the synthesizer onto six loops, with four notes per loop.

So what should the notes be? I wrote a little sequencer and started writing lists of notes. I tried ordering the notes in basic intervals (root, 3rd, tonic, etc) and I tried random ordering. Eventually I settled on this type of “minimal transposition” scheme where you order notes by grouping close notes together. For example: first suppose you chose four chords: Am, F, C, G. acrostic will first determine the notes for each chord in a separate column:

Am  F   C   G  
---------------
A   F   C   G
C   A   E   B
E   C   G   D

Then I would rearrange the notes of each chord in each column so that there are minimal changes between different columns.

Am  F   C   G  
---------------
C   C   C   D
A   A   G   G
E   F   E   B

Then I would read off, row by row, the notes where each row would be one full loop in my looper script. This is not a new techniique, in fact its a very simple form of voice leading. Eventually these long lists of notes where turned into songs when I put everything together. For example, this list of notes became this song:

A realization about articulation

I recorded the songs with that toy synthsizer two years ago. Since then I’ve been thinking about it - how simple the process was for generating the music and how much I enjoyed the result. Part of the result, though, was not in the pitches themselves but in their articulation - i.e. how loud one series of notes might be.

So I went ahead to write another program. One that would automatically create the matrix of notes and record each row of the matrix, one note at a time, into different loops. I called this program “acrostic” since the note matrix has meaning in its columns (chords) and in its rows (melodies). The key though, was to provide each loop with a specific articulation - basically it would allow each one to get louder and softer randomly over time. This allows different meldies to come in and out of focus, which instantly gives songs their “movement”.

For example, here is a track from my latest album “generations” where the volume of each track is visualized by the vertical position of one of the filled circles:

To me, this sounded great. Not only that, using the same four chords would give greatly different results which is an important part of generative music (to me). The randomness in the process was enough to generative varied performances but not too much to produce noise.

The recipe for generative music

So after this little discovery I had a basic recipe for generative ambient music.:

1. Choose chords.
2. Create matrix of notes from chords and rearrange rows in each column to minimize distance between columns.
3. Render each row, one note at a time, with some instrument/synthesizer into a looper that stores each row as its own track.
4. Add articulation to each loop track by modulating its volume with a simple LFO.

I used this to set about recording dozens and dozens of songs for this album. It was easy to record - in my acrostic software, I basically just put the chords I wanted (as input) and then let it go while capturing the output. Afterwards I did a little bit of mastering (normalizing levels, taking off some of the low end) and released it as “generations”. There is one of the title tracks with an animation I made visualizing the articulation of tracks:

Each track is made the same way - following those four steps above - and using the same sound source (a synthesizer called the Make Noise Strega). But you could easily do the same thing with a toy synthesizer, or any instrument really. There are some other little tricks - I introduced a little variation by adding in random melodies that where interpolations between neighboring columns in that matrix. But that’s something for another story!

My code referenced

Here is a list of repositories I wrote that were mentioned above:

• miti - a text-based music sequencer
• oooooo - a six-loop sampler
• acrostic - a chord sequencer and sampler

Publicizing internet streams without port forwarding

For me, the hardest part about making an internet radio is opening it up to the public internet. I don’t have access to port-forwarding in many scenarios where I want to broadcast (i.e. broadcasting from friends places, or from a music device). My solution to the problem of broadcasting audio without port-forwarding was to create a free and public “dummy” server that handles any incoming traffic and forwards to any number of connected clients. This server keeps track of connected clients and routes incoming packets to them, otherwise discarding the packets.

You could then simply use a common utility like curl to stream data to this server to those peers. This is essentially what Twitch/Youtube/Instagram do, but I designed a server without any social aspects, its just a dumb URL endpoint that can consume and produce audio streams (or any stream).

The server itself lives at broadcast.schollz.com. The code is open-source and available on Github at schollz/broadcast-server. The code is less than 400 lines, and exceedingly simple. You POST data streams to any endpoint you want, and processing GET requests to that endpoint just return the data in that stream. There are special flags if you want to advertise or archive your stream.

Now using this really dumb server which just lives on a DigitalOcean droplet you can easily create internet streams from anywhere, with anything, using a variety of methods.

Live station using ffmpeg

To make a live internet radio from the input of your computer, you can do it in one line (albeit long line):

1ffmpeg -f alsa -i hw:0 -f mp3 - | \
2    curl -s -k -H "Transfer-Encoding: chunked" -X POST -T - \
3    "https://broadcast.schollz.com/YOURSTATIONNAME.mp3?stream=true&advertise=true"

That one-liner takes input from your default input hw:0 and streams it into a POST request to the dummy server making the stream available at broadcast.schollz.com/YOURSTATIONNAME.mp3. Make sure you wear headphones if you create a stream on the same computer you are listening to the stream (because feedback)!

Live station using vlc

Linux:

1vlc -I dummy alsa://plughw:0,0 --sout='#transcode{vcodec=none,acodec=mp3,ab=256,channels=2,samplerate=44100,scodec=none}:standard{access=file,mux=mp3,dst=-}' --no-sout-all --sout-keep | curl -k -H "Transfer-Encoding: chunked" -X POST -T -  'https://broadcast.schollz.com/YOURSTATIONNAME.mp3?stream=true&advertise=true'

Mac (requires VLC 2.0.0 or later):

1vlc -I dummy -vvv qtsound:// --sout='#transcode{vcodec=none,acodec=mp3,ab=256,channels=2,samplerate=44100,scodec=none}:standard{access=file,mux=mp3,dst=-}' --no-sout-all --sout-keep | curl -k -H "Transfer-Encoding: chunked" -X POST -T -  'https://broadcast.schollz.com/YOURSTATIONNAME.mp3?stream=true&advertise=true'

Playlist station using ffmpeg

Creating an internet radio station from a music playlist is also easily done. You need one extra common utility: cstream which will make sure the stream is sent in realtime, so its not loaded into memory. Here’s a one liner that will take any folder and create a randomly shuffled playlist and stream it to the station:

1find ~/Music/ | grep 'wav\|mp3\|flac' | shuf | sed -e "s/^/file '/" | \
2    sed -e "s/$/'/" > /tmp/playlist.txt && \
3    ffmpeg -f concat -safe 0 -i /tmp/playlist.txt -f mp3 -ar 44100 -b:a 256k - | \
4    cstream -t 32k | \
5    curl -s -k -H "Transfer-Encoding: chunked" -X POST -T -  \
6    "https://broadcast.schollz.com/YOURSTATIONNAME.mp3?stream=true&advertise=true"

In this case we are streaming at 256 kbps, which is 32 KB/s, so cstream is set to throttle the data to that rate so it transfers in “realtime”.

Play file using vlc

1vlc -I dummy ANYFILE ':sout=#transcode{acodec=mp3,ab=52,channels=2}:standard{access=file,mux=mp3,dst=-}'  | cstream -t 16k | curl -k -H "Transfer-Encoding: chunked" -X POST -T -  'https://broadcast.schollz.com/YOURSTATIONNAME.mp3?stream=true&advertise=true'

Method 2 of 2: Broadcasting using icecast2 and darkice

The icecast2 and darkice utilities are installed via apt-get and really powerful when it comes to streaming. I won’t go into much detail with these two, because there are a lot of guides for it at other places. Here is a great guide for doing live radio on a Raspberry Pi.

If you are on Linux, you can also make your entire computer audio part of the stream (i.e. your browser, VLC, etc.). You can do this with JACK. Simply install JACK:

1sudo apt install jackd pulseaudio-module-jack 

Then start JACK and load the sink/source for pulse audio:

1jackd -R -dalsa -dhw:0,0 &
2pactl load-module module-jack-sink channels=2
3pactl load-module module-jack-source channels=2
4pacmd set-default-sink jack_out
5pacmd set-default-source jack_in

Now start icecast2 and darkice with a JACK configuration.

1wget https://raw.githubusercontent.com/schollz/broadcast/main/icecast.xml
2wget https://raw.githubusercontent.com/schollz/broadcast/main/darkice.cfg
3icecast2 -c icecast.xml & 
4darkice -c darkice.cfg

Now simply connect darkice into JACK!

1jack_connect 'PulseAudio JACK Sink:front-right' darkice:right
2jack_connect 'PulseAudio JACK Sink:front-left' darkice:left

Making a darkice stream public

Once you have your darkice/icecast2 stream, you can make it public by port forwarding. There are lots of guides for that. But another way you could do it is using the dummy broadcast server I mention above. Simply pipe the output from darkice and POST it to the server!

1curl http://localhost:8000/radio.mp3 | \
2    curl -H "Transfer-Encoding: chunked" -X POST -T -  \
3    'https://broadcast.schollz.com/YOURSTATIONNAME.mp3?stream=true'

That’s it! Now your stream will be available at broadcast.schollz.com/YOURSTATIONNAME.mp3.

Integration with live music tools

I do a lot of projects with the pi-based sound computer called “norns” which lets you define its behavior with scripts and DSP and basically functions as a sample-cutter, polysynth, drum machine, drone box, granulator, etc.

The norns music tool is based in Linux, so the above methods are just as applicable. This means that the norns can be turned into a broadcast station, while retaining all the functionality as a DSP device! Here’s my little script that converts live audio into a broadcast, directly from your house or live music venue (as long as you have WiFi).

The above norns script uses the darkice/icecast2 because it is based around JACK, but it works beautifully despite running on a Raspberry Pi 3+ and simultaneously running a sample-cutter and SuperCollider synthesizers.

Pianists are lucky. The piano is an instrument that - unlike trumpets, saxophones, etc. - sometimes is available to play for free. These free-to-play pianos are called “public pianos” (or “street pianos”) which often appear in airports, train stations and parks during good weather.

Pianists know that these public pianos exist, no one knows where most of these pianos are and as such they exist in the world as a sort of magical entity whose secret location needs to be discovered. I am attempting to discover the location of all of them, and make a map of every piano in the world.

The result of this attempt is a website: Pianos for Travelers. This blog post tells about how the website was made.

Getting the data

I estimate that there are thousands of public pianos in the world and at least one in every major city in the world. Obviously I cannot go to every city in the world to find every piano, so I am attempting to use the internet to find their locations.

There is a handful of websites that compile piano lists including a slowly updated wiki and an out-of-date map, forums, subreddits and etc. I collected hundreds of piano locations going through these types of websites and manually cataloging their coordinates in a file.

After going through every website I could find, I only had a few hundred pianos - there were still over a thousand out there. Luckily, I realized that social media actually has tracked locations of people using the hashtags #publicpiano and I was able to use APIs to collect their coordinates as well. This netted hundreds of more pianos.

At this point, until I find another source of pianos, I will be bootstrapping myself and heeding user input to gather the locations of more pianos. This is usually where the other websites failed, and mine might too, so I’m still looking for alternatives to this practice.

Creating the website

I designed the Pianos for Travelers site with my good friend. We used a very simple stack - Go std-lib http router, with Postgres 12 backend, JQuery v3 frontend, and using Tachyons for CSS. (I actually disagree that Postgres is very simple, but it is very powerful when it comes to GIS information).

The Go language constantly impresseses me with how fast we can move into production - for example we needed a CAPTCHA and found an amazing package by dchest that was basically a dropin into our std-lib web server. Same thing happened when we switched to Websockets. We found the entire site took less than two weeks to build (and we only spent our free time on it).

Shipping the website

The website, Pianos for Travelers, is live now. We posted about it on sites that might have interest in places with pianos (Piano forums, subreddits) and it gained a little traction. It is up for perpetuity now, though so I’m hoping it provides some useful information to the people that find it.

One of the things that gave it a distinctive sound was the phase distortion synthesis. Phase distortion synthesis is similar to phase modulation and was used to simulate a resonant filter. This is super useful to be able to make iconic filter sweeps.

Basically it was done using a digital hard sync, by multiplying a hard synced signal against the inverted counter to level out the final signal. It is well described in Casio Computer Co Ltd patent filed in 1983:

It turns out it’s quite easy to generate this plot, pretty much exactly, just using SuperCollider code.

(
{
	var freqBase=100;
	var freqRes=130;
	var pdbase=Impulse.ar(freqBase);
	var pd=Phasor.ar(pdbase,2*pi*freqBase/s.sampleRate,0,2*pi);
	var pdres=Phasor.ar(pdbase,2*pi*freqRes/s.sampleRate,0,2*pi);
	var pdi=LinLin.ar((2*pi-pd).max(0),0,2*pi,0,1);
	[pd/(2*pi),pdres/(2*pi),SinOsc.ar(0,pdres),pdi,Lag.ar(SinOsc.ar(0,pdres)*pdi,1/freqBase)];
}.plot(0.011)
)

The code above results in a SuperCollider plot that looks eerily similar to the patent figure.

Now we can take that code and put it into a synth, and we have a re-creation of one small part of the Casio keyboard from 1984.

(
Routine {
	SynthDef("casioish",{
		arg freq=220, amp=0.5;
		var freqBase=freq;
		var freqRes=SinOsc.kr(Rand(0,0.2),0).range(freqBase/2,freqBase*2);
		var pdbase=Impulse.ar(freqBase);
		var pd=Phasor.ar(pdbase,2*pi*freqBase/s.sampleRate,0,2pi);
		var pdres=Phasor.ar(pdbase,2*pi*freqRes/s.sampleRate,0,2pi);
		var pdi=LinLin.ar((2pi-pd).max(0),0,2pi,0,1);
		var snd=Lag.ar(SinOsc.ar(0,pdres)*pdi,1/freqBase).dup;
		snd=Splay.ar(snd);
		snd=snd*EnvGen.ar(Env.perc(0.005,10));
		Out.ar(0,snd*amp);
	}).add;
	s.sync;

	x = Synth("casioish",[\freq,60.midicps]);
	y = Synth("casioish",[\freq,62.midicps]);
	z = Synth("casioish",[\freq,65.midicps]);
	w = Synth("casioish",[\freq,60.midicps/2,\amp,1.0]);
}.play
)

And here’s an example of the resulting sound:

The tiny hack I want to introduce is so simple but adds a great deal of functionality to the PO-32. The PO-32 drum sequencer has a unique feature where the four columns are designated as different channels and while playing, any key in will mute that corresponding channel. This is extremely useful in case you make a drum pattern with four instruments and want each one to come in one at a time. To use it though, you physically have to hold down one key for each channel you want to mute.

What does this tiny hack do?

This tiny hack bypasses your fingers and uses switches. The switches are placed at the bottom, out of the way of most of the keys, and simply allow you to activate the “mute channel” feature without holding down any keys. Since its placed in parallel of the button, there is no other circuitry needed. Here’s a demo of it:

Why do this tiny hack?

This tiny hack is useful if you use the PO-32 and find yourself making a pattern for individual instruments: i.e. one pattern with bass drum, another pattern with the same bass drum but adding a snare, etc. This tiny hack lets you build a single pattern with all the instruments and you can toggle them on/off simply by flipping a switch!

How do I do this tiny hack?

One note of caution: I have confirmation from Teenage Engineering that this will void your warranty.

This tiny hack is tiny indeed. All you need are four switches, some wire, and a soldering iron (and a PO-32 of course).

First connect the switches to two wires and cut them so that they are similar length (will make it easier for next step).

Then heat up the bottom of one of the legs of the key on the PO-32 and use solder to solder the wire of the switch to it. Repeat for each key that you want to toggle!

One note on use: make sure you turn off the switches when you are done playing with the PO-32 otherwise it will continue to remain on until the battery expires.

I made this specifically for the upcoming flash crash performance. this script is a cross between a live coding environment and a tracker - its sorta a tracker you sequence with code. the code you can sequence is any norns lua code. using code you can access any of the standard norns features (midi, osc, crow), and I’ve also added interfaces to “lite” versions of several of my own scripts (like oooooo, supertonic, amen, and mx.samples). some basic features of internorns:

• sequence pitches with text, in chords (e.g. Cm7/Eb) or notes (e.g. a4 eb3)
• sequence midi devices and add cc lfos
• sequence crow with pitches, automatically generating voltage/envelope
• play/rec into three stereo loops via softcut
• built-in drum synth that can be sequenced/modulated in realtime
• built-in sample player that can be quantized
• sequence notes/chords from any instrument in mx.samples
• special tape stop / start global fx

internorns works quite simply: it runs an internal sequencer that runs code. there is an internal clock that makes 4 steps per beat, and 4 beats per measure. at each step it checks to see if there is code to run in that step in the current measure and attempts to run it. code can be added to steps using the built-in functions.

See data/getting-started.lua to get started and preview how it works. the process is music.

Requirements

• computer
• norns
• midi device (optional)
• crow (optional)

Documentation

start the internorns script on norns.

now choose an editor to live-code, I suggest usig either maiden (in the browser), visual studio code, or vim. instructions for each are below.

wget https://github.com/schollz/wscat/releases/download/binaries/wscat
chmod +x wscat
sudo mv wscat /usr/local/bin/

1set underline
2nnoremap <C-c> <esc>:silent.w !wscat<enter>
3inoremap <C-c> <esc>:silent.w !wscat<enter>i

open up dust/data/internorns/getting-started.lua in your editor to learn how to use internorns.

Install

make sure you install all of the following (or update if you already have them):

;install https://github.com/schollz/mx.samples
;install https://github.com/schollz/supertonic
;install https://github.com/schollz/internorns

then restart your norns.

https://github.com/schollz/internorns

(info on what is happening in video)

Introspection

This drum machine introspects by looking at any of the current drum patterns and generating a new drum pattern based specifically on that pattern and which instruments they are (e.G. A snare rhythm based on the kick pattern).

These generative rhythms are accomplished using google’s “Variational autoencoder” for drum performances. their blog post explains it best (and their paper explains it better), but essentially they had professional drummers play an electronic drum-set for 12+ hours which was later used to feed a special kind of neural network. I used their model from this network and sampled it randomly to produce “New” groups of drum rhythms (>~1,000,000 of them). Then I created probability distributions from calculating bayesian probabilities from each instrument to each other instrument, within each rhythm grouping. This probability table can then generate a snare drum pattern based on a kick drum pattern, or generate a hihat pattern based on a snare drum pattern, etc. Etc.

Sounds

The sounds for this drum machine come from a new engine which I call “Supertonic” because it is a as-close-as-I-can port of the microtonic vst by soniccharge.

The act of porting is not straightforward and the experience itself was a motivation for this script - it helped me to learn how to use supercollider as I tried to match up sounds between the vst and supercollider using my ear. I learned there is a lot of beautiful magic in microtonic that makes it sounds wonderful, and I doubt I got half of the magic that’s in the actual vst (so this is by no means a replacement). Looking at the resulting engine you might notice some weird equations that are supposed to be approximating the magic behavior in the true microtonic. This script also includes a standalone supercollider drum machine to use with this engine (and conversion scripts to convert microtonic patches).

Here is a demo comparing microtonic and this engine.

Drummer in a box

In the end, this script is a little drum machine in a box and also a new drum machine engine for norns, a little like @21echoes’s cyrene, @pangrus’s hachi, or @justmat’s foulplay.

For me personally, this script is an experiment. To try to answer the question: what is it like to perform with an ai generated rhythm section (I.E. Paralleling what its like to play with a ai generated piano)? Is it good? Surprisingly so, sometimes.

Requirements

• norns

Documentation

all the parameters for the engine are in the PARAM menu, as well as preset loading.

on the main screen:

• K2 starts/stops
• K3 toggles hit
• E2 changes track (current is bright)
• E3 changes position in track
• hold K1 and turn E3 to erase

this script automatically detects all midi keyboards and will start/stop based on midi start/stop events.

you can hold K3 and move E2 to lay down a lot of beats.

introspection

introspection requires downloading a prior table (~100 mb, not included in repo) and sqlite3. both of these can be installed by running this command in maiden:

os.execute("sudo apt install -y sqlite3; mkdir -p /home/we/dust/data/supertonic/; curl -L --progress-bar https://github.com/schollz/supertonic/releases/download/v1_ai/drum_ai_patterns.db > /home/we/dust/data/supertonic/drum_ai_patterns.db")

once installed, you have some new button combos available:

• hold K1, then press K3 to generate a new pattern based on the highlighted pattern
• hold K1 and turn E2 to change the highlighted pattern (basis of the generation)
• hold K1 and press K2 to generate beats 17-32 based on beats 1-16 for current instrument

using your own microtonic presets

if you have microtonic you can easily use your own microtonic preset. simply copy your microtonic preset file (something like <name>.mtpreset) and and save it into the /home/we/dust/data/supertonic/presets directory. then, you can then load these presets via the PARAM > SUPERTONIC > preset menu.

converting microtonic presets for use with SuperCollider

you can also use the engine directly with SuperCollider. the engine file is synced with a SuperCollider script, lib/supertonic.scd. an example drumset is in lib/supertonic_drumset.scd. you can easily get a SuperCollider file with your microtonic presets by running this lua script:

lua ~/dust/code/supertonic/lib/mtpreset2sc.lua /location/to/your/<name>.mtpreset ~/dust/data/supertonic/default.preset > presets.sc

known bugs

the supertonic engine is pretty cpu-intensive, so if you have 4-5 instruments all doing fast patterns (or fast tempo) you will hit cpu walls and hear crunching. any ideas to improve cpu usage are welcome :)

the pattern generation (k1+k3 or k1+k2) runs asynchronously but I’ve noticed that sometimes it might cause a little latency when using it while performing (generating patterns while playing).

if you aren’t seeing any new randomly generate patterns when pressing K1+K3/K2, it could be that the pattern that you’re using as a basis doesn’t exist in the database (and therefore won’t produce any new patterns).

thanks

the ex-dash patterning functions are from @license from the collaborative song project. the flying confetti is from @eigen’s brilliant pico-8 wrapper. also thanks @dan_derks, our little discussion helped me figure out the beginnings of this thing. finally, big big thanks to @midouest who shared their microtonic supercollider project which showed me some tricks I had missed and also showed me I was on the right track (because our implementations had a lot of parallels).

Download

install via maiden or

;install https://github.com/schollz/supertonic

make sure to restart after installing because it includes a new engine.

note, that to use the introspection you must also install the probability database (instructions here).

https://github.com/schollz/supertonic

with wobblewobble you can…

• assign each of the four crow outputs one of many different types of slowly oscillating modulation sources.
• configure crow as midi->CV or midi->envelope atop any oscillation (or none)
• visualize slow oscillations from crow inputs (<7hz)

this script was also designed to be a library (all functionality is available in the menu) so that it can be imported for crow functionality from other scripts. no examples of this yet, though.

Requirements

• norns
• crow

Documentation

from the main norns screen you can:

• hold K1 and K2/K3 to switch crow
• press K2/K3 to switch modulator
• E1 changes frequency
• E2/E3changes lfo min/max
• K1+E2 or K1+E3 changes meta

the params menu has more options, including:

• clamping the absolute minimum and maximum values
• add midi input to control pitch (top note, any note) or add an envelope
• meta params for the modulators
• change slew rate

oscillators are configured in SuperCollider making it easy to configure / design / add new oscillators, including complex ones from the chaotic UGens. current oscillators:

constant

sine

triangle

wobbly sine

snek

lorenzian

henonian

random walk

latoocarfian

standard chaotic

fbsine

quad

gingerbreadman

many many thanks to @proswell for the excellent additions (grid support, five more modulators, adding inputs, adding metas) as well as testing.

any other PRs are welcomed with open wings.

Install

;install https://github.com/schollz/wobblewobble

I love the po-33 and for a long time I’ve wanted something similar for norns (e.g. a micro-sampler/splicer + sequencer). I tried making something similar in abacus but I didn’t have a grid so the ux was very limited and ultimately gave me a bad flow. after making amen which is a great looper+fx for me, I realized that a lot of my work could be re-used to make a splicer+sequencer too. instead of designing a ux from scratch like I did for abacus, I decided to just copy the ux from the po-33 to have complete skill transferability between instruments (much like you would have between two pianos or two saxophones). however there are some key differences:

• thirtythree is based in the norns which has higher sound quality (48khz, 24bit, stereo samples (though higher is not always better))
• thirtythree fx are sound-specific instead of global (see PARAMS menu to toggle global fx in the po-33).
• thirtythree fx stack (except looping fx) so multiple fx can be applied simultanouesly.
• in addition to recording from line-in, you can also load a file into any bank (see PARAMS menu to toggle this bheavior).
• you have two choices of layouts (see the PARAMS menu) - the classic 5x5 type layout and a compressed 4x6 layout that lets you stamp more of the thirtythree apps across the grid.
• you can chain up to four operators on a grid, and there is ~10 note polyphony shared across all of the operators.
• thirtythree doesn’t save instantaneously like the po-33 does. you can save manually using the key combo (below) or wait for the auto-save to occur (which occurs after idling for ~3 seconds).
• thirtythree dumps can be shared via the norns.online cloud.

requirements

• norns
• monome grid or most midi grids

documentation

the official te guide for the po-33 explains the usage for this app as thirtythree follows all of the same key combos. basics:

• melodic (buttons 1-8) and drum splicing (buttons 9-16)
• record with record+1-16 (option to load files instead in parameters)
• select sound with sound+1-16
• write mode activated with write
• select pattern with pattern+1-16
• play pattern with play
• toggle parameters with fx
• adjust tone (pitch+volume) with E2 and E3
• adjust filter (lp+hp) with E2 and E3
• adjust trim (start+end) with E2 and E3
• delete sound with record+sound
• (not implemented) copy slice with write+sound+9-16+1-16
• add effects with fx+1-16
• change swing with bpm+K3
• change tempo tapping bpm
• change tempo with bpm+K3
• change master volume with bpm+1-16
• parameter locking with write+(K2 or K3)
• chain pattern with pattern+1-16
• copy pattern with write+pattern+1-16
• clear entire pattern with record+pattern
• (not implemented) clear current sound pattern with record+sound+pattern (new)
• backup data with write+sound+play
• restore data with write+sound+record

when in the “trim” mode, you can use E2 or E3 to jog the endpoints. use E1 to zoom in to the last endpoint moved. in the tone or filter mode, you can adjust things with just E2 or E3.

here are the two possible layouts available to stamp the grid with:

thirtythree effects

the fx are the same, except unison effects replaced by stereo auto-panning and a bitcrush. some fx stack (looping / scratching do not).

1. loop 16
2. loop 12
3. loop short
4. loop shorter
5. unison autopan
6. unison low bitcrush
7. octave up
8. octave down
9. stutter 4
10. stutter 3
11. scratch
12. scratch fast
13. 6/8 quantize
14. retrigger pattern
15. reverse
16. none

saving / loading

thirtythree automatically saves your current state when you are idle - this is the “default” that will be loaded each time you open thirtythree (just like when you turn on a po-33). you can also use the key combo to save (see above).

in addition to automatic saving, you can save your current state to a separte file using PSET > SAVE and load with PSET > LOAD.

thirtythree also optionally lets you share your work with others. first install and run norns.online:

;install https://github.com/schollz/norns.online

once you run that app and choose a username, you can then use thirtythree to upload+download shares. there will be a new menu PARAMS > SHARE > upload/download which you can use to share or backup your work to the cloud.

known limitations/bugs

• backups will not restore if you move/delete the ~/dust/audio/thirtythree audio folder, where all the audio data is stored.
• looping fx don’t stack.
• sometimes there is race condition in the bpm, if you get wacko pattern stepping, restart. so far I haven’t been able to reproduce consistently.
• using 6/8 beat fx might cause operators to get out of sync. if this happens, stop all the patterns and start them again and they should sync back up.

thanks

thank you @license and @catfact for never ceasing to teach me a half dozen supercollider tricks in half as many lines of code. thanks to @proswell and @CrazyEmporer893 for beta testing. thanks to @eigen for the p8 library which was a jumping off point for the graphics and source of the the manga. thanks to @glia for allow me to include a yelidek kit as the default drum kit. thank you @jredou_ko for help with the graphics.

install

install with

;install https://github.com/schollz/thirtythree

the first time you run thirtythree it will update your norns with the aubio library which is used to generate default onsets. this might take 10-15 seconds.

SuperCollider is a free platform for audio synthesis and algorithm composition. It really excels at real-time analysis, synthesis and processing and provides a framework for seamless combination of audio techniques like additive and subtractive synthesis (as explored in my previous tutorial on drones) or be used for sophisticated sampling+splicing, the focus of this tutorial.

A sampler+splicer is useful when you have an audiofile and you want to cut it up into pieces and play it back in specific order. You can do this and change the tempo of a song by playing back snippets faster, or shuffle drum breaks to make new beats, or map specific sounds from a single audio file.

Before you begin

Before starting, make sure you have SuperCollider and plugins (if you want).

• Install SuperCollider
• Install SuperCollider plugins (optional, but recommended)

If you have these basic things installed, you are good to go. Feel free to do a SuperCollider tutorial if you want, but I’ll assume you don’t know how to use it.

With SuperCollider installed, you will need to run the program and then start the server using Ctl+B.

Here is a link to download all the files for this tutorial: https://github.com/schollz/sc-tutorial/archive/refs/heads/main.zip

1. Load a buffer with audio

We are going to start playing with a simple piano loop that I got from the radio:

Let’s first load this into SuperCollider, by running the following lines:

(
  b=Buffer.read(s,thisProcess.nowExecutingPath.dirname++"sounds/pianochords.wav");
)

2. Use PlayBuf to play sample

And now lets play it back using the simple PlayBuf “UGen”:

(
SynthDef("PlayBufPlayer", {
    arg out = 0, bufnum = 0;
    var snd;
    snd=PlayBuf.ar(2, bufnum, BufRateScale.kr(bufnum), doneAction: Done.freeSelf);
    Out.ar(out,snd)
}).play(s, [\out, 0, \bufnum, b]);
)

When running that code you should immediately hear it play back!

While this already works as a sampler (it plays back a sample), it actually won’t let us set the endpoints so we can splice the sample. Let’s do that next.

3. Use BufRd + Phasor to precisely play sample

In order to precisely control the position of playback we must use something called a Phasor. A Phasor is basically a sawtooth-like signal that increments at a specific rate. It is really useful as a position index, to set the position of playback, because you can control its start and end poitns and the rate.

Here is what a Phasor will look like:

And instead of going from 0-100 we can change it to whatever we want while also changing the rate:

We will introduce a lot of code to do this, but I’ll go through it piece by piece:

(
x=SynthDef("PlayBufPlayer", {
    arg out=0, bufnum=0, rate=1, start=0, end=1;
    var snd,pos,frames;

    rate = rate*BufRateScale.kr(bufnum);
    frames = BufFrames.kr(bufnum);

    pos=Phasor.ar(
        rate:rate,
        start:start*frames,
        end:end*frames,
        resetPos:start*frames,
    );

    snd=BufRd.ar(
        numChannels:2,
        bufnum:bufnum,
        phase:pos,
        loop:0,
        interpolation:4,
    );
    Out.ar(out,snd)
}).play(s, [\out, 0, \bufnum, b]);
)

The rate is now an argument and default is 1. However, it gets modulated by BufRateScale.kr(bufnum) where bufnum is a reference to the audio file. This modulation is a scale based on the sample rate of the audio file and the sample rate of the server. In the case the two sample rates don’t match, then you do need to change the speed you playback a sample to make sure it is exactly the same pitch when played at the other sample rate at a rate of 1.

The Phasor has start and end points and a rate and a resetPos which is the position it resets. The Phasor takes inputs in numbers of frames so we multiple the start and end points (which I am denoting to always be between 0 and 1) by the number of frames.

The BufRd is where the playback happens. We set the position of playback using the phase parameter. Notice we set loop to 0, but in actuality, because we are using a Phasor this will be ignored.

We can now change the start and end positions by running this command after running the SynthDef above:

x.set(\start,0.5,\end,0.6);

One thing you might notice is that you cannot get this sample to reset if you run that command more than once. Resetting is a really useful thing, and we can fix this SynthDef to allow that.

4. Use trigger to allow resetting sample

This fix is only one line, and adding a new argument, t_trig. This is a special argument and when it goes from 0 to 1 it can cause a change in a UGen. In this case we will use it to cause a change in Phasor, which will make it reset.

(
x=SynthDef("PlayBufPlayer", {
    arg out=0, bufnum=0, rate=1, start=0, end=1, t_trig=0; // NEW t_trig!
    var snd,pos,frames;

    rate = rate*BufRateScale.kr(bufnum);
    frames = BufFrames.kr(bufnum);

    pos=Phasor.ar(
        trig:t_trig, // NEW!
        rate:rate,
        start:start*frames,
        end:end*frames,
        resetPos:start*frames,
    );

    snd=BufRd.ar(
        numChannels:2,
        bufnum:bufnum,
        phase:pos,
        loop:0,
        interpolation:4,
    );
    Out.ar(out,snd)
}).play(s, [\out, 0, \bufnum, b]);
)

Now you can send various commands to this SynthDef which can cause it to reset to the new start position:

x.set(\t_trig,1,\start,0,\end,1)
x.set(\t_trig,1,\start,0.5,\end,1)

You can hear in that audio that I reset the position multiple times.

One problem that persists though is that, as mentioned earlier, we are using a Phasor which loops forever and overrides the loop parameter of the BufRd UGen. It would be better to have an option to set the loops to any number you want.

5. Use an envelope, Env, to control looping

It turns out we can do a little trick to get a precise loop - we can use an envelope. An envelope is essentially a curve in time that we can multiple by the output amplitude. The function of the curve is to shape the amplitude, and in this case we will shape it so it turns on and then turns off after a certain amount of time.

A basic envelope is generated like this:

({ 
EnvGen.ar(
    Env.new(
        levels: [0,1,1,0],
        times: [0,1,0],
    ),
)
}.plot(2);)

which generates an envelope that lasts 1 second and then goes to 0. It looks like this:

There are all sorts of envelopes you can make. We can make this one fancier by having it fade in/out a little smoother by adding some time to the “attack” and “release” part of the envelope:

(
{ EnvGen.ar(
    Env.new(
        levels: [0,1,1,0],
        times: [0.2,1-0.4,0.2],
        curve:\sine,
    ),
)
}.plot(2);
)

Great! Well all we need to do to preserve a loop is to calculate how long the duration of the envelope should be. This involves a little math, but it should be related to the number of frames, the rate (1/2 the rate = twice the time), the sample rate, and the number of loops:

duration = frames*(end-start)/rate.abs/s.sampleRate*loops

We then just need to trigger the envelope when we trigger our sample and voila!

(
x=SynthDef("PlayBufPlayer", {
    arg out=0, bufnum=0, rate=1, start=0, end=1, t_trig=1,
    loops=1; // NEW
    var snd,pos,frames,duration,env;

    rate = rate*BufRateScale.kr(bufnum);
    frames = BufFrames.kr(bufnum);
    duration = frames*(end-start)/rate/s.sampleRate*loops; // NEW

    // envelope to clamp looping
    env=EnvGen.ar(
        Env.new(
            levels: [0,1,1,0],
            times: [0,duration-0.01,0.01],
            curve:\sine,
        ),
        gate:t_trig,
    );

    pos=Phasor.ar(
        trig:t_trig,
        rate:rate,
        start:start*frames,
        end:end*frames,
        resetPos:start*frames,
    );

    snd=BufRd.ar(
        numChannels:2,
        bufnum:bufnum,
        phase:pos,
        interpolation:4,
    );

    snd = snd * env;

    Out.ar(out,snd)
}).play(s, [\out, 0, \bufnum, b]);
)

Now we can set the loop number, and reset the sample, and control the start and stop positions:

x.set(\t_trig,1,\start,0.0,\end,0.15,\loops,1)
x.set(\t_trig,1,\start,0.5,\end,0.52,\loops,6)

This sounds like the following:

You can see that loops stop and stay stopped! And you can make it loop as many times as you want.

But what happens if we reverse? If you change the rate to -1 it actually won’t play:

x.set(\t_trig,1,\start,0.5,\end,0.6,\loops,3,\rate,-1)

Its because the start and end points need to swap. If we do that, then we can succesfully play in reverse:

x.set(\t_trig,1,\start,0.6,\end,0.5,\loops,3,\rate,-1)

But, lets alter the code above so that it works in reverse without having the start and end points switching around!

6. Automatically swap start/end points when reversing

To swap the start/end points we need to check the rate and add some logic about whether the rate is positive or negative. You can’t really use if-statements in SuperCollider (well you can but its weird), but we can do this easily with math. For example, the start parameter of the Phasor can calculate a binary whether the rate is positive or negative and then make a sum based on it:

start:(((rate>0)*start)+((rate<0)*end))*frames

It looks complicated but its quite simple. If the rate>0 then the second part in the parentheses is 0 and its just start. If the rate<0 then the first part is 0 and then it just switches to end.

The final SynthDef looks like this:

(
x=SynthDef("PlayBufPlayer", {
    arg out=0, bufnum=0, rate=1, start=0, end=1, t_trig=1,
    loops=1;
    var snd,pos,frames,duration,env;

    rate = rate*BufRateScale.kr(bufnum);
    frames = BufFrames.kr(bufnum);
    duration = frames*(end-start)/rate.abs/s.sampleRate*loops; // use rate.abs instead now

    // envelope to clamp looping
    env=EnvGen.ar(
        Env.new(
            levels: [0,1,1,0],
            times: [0,duration,0],
        ),
        gate:t_trig,
    );

    pos=Phasor.ar(
        trig:t_trig,
        rate:rate,
        start:(((rate>0)*start)+((rate<0)*end))*frames,
        end:(((rate>0)*end)+((rate<0)*start))*frames,
        resetPos:(((rate>0)*start)+((rate<0)*end))*frames,
    );

    snd=BufRd.ar(
        numChannels:2,
        bufnum:bufnum,
        phase:pos,
        interpolation:4,
    );

    snd = snd * env;

    Out.ar(out,snd)
}).play(s, [\out, 0, \bufnum, b]);
)

And when we use the problematic statement above, we can see that it just works:

x.set(\t_trig,1,\start,0.5,\end,0.6,\loops,3,\rate,-1)

Okay, now one more thing. You may have noticed in playing around that there is an annoying “click” or “pop” when you reset the sample. For example, try to send the following a bunch of times really fast you’ll hear an audible noise artifact from the switching:

x.set(\t_trig,1,\start,0.2,\end,0.6,\loops,1,\rate,1)

It’s unpleasant right? Well we can make one more improvement to remove this type of thing.

7. Preventing switching pops

The main problem of pops caused by switching is that there isn’t a smooth transition to the next position. The smooth transition we expect actually needs to come from the audio itself and not the position. To make a smooth transition then, we need to crossfade between the old loop and the new loop.

Doing this is not nearly as hard as it seems at first. We already have one loop playing. We need to make another loop. Then we just need to make a new trigger, based on t_trig which we already use, that flips between triggering the two loops. We can use a special UGen for this called ToggleFF and then another UGen called Latch which will allow us to change only one of the loops at a time, alternating between the two.

Below is the new code. We still have the same arguments, but we have new variables for the two loops (startA+B, endA+B, crossfade, and aORB which is the new trigger). The crossfade is doing the major work here, we can create the audio crossfade by using something called Lag which will transition between the two values slowly.

For example, this is a 50 ms crossfade:

crossfade=Lag.ar(K2A.ar(aOrB),0.05);

This is what the new code ends up being:

(
x=SynthDef("PlayBufPlayer", {
    arg out=0, bufnum=0, rate=1, start=0, end=1, t_trig=0,
    loops=1;
    var snd,snd2,pos,pos2,frames,duration,env;
    var startA,endA,startB,endB,crossfade,aOrB;

    // latch to change trigger between the two
    aOrB=ToggleFF.kr(t_trig);
    startA=Latch.kr(start,aOrB);
    endA=Latch.kr(end,aOrB);
    startB=Latch.kr(start,1-aOrB);
    endB=Latch.kr(end,1-aOrB);
    crossfade=Lag.ar(K2A.ar(aOrB),0.05);


    rate = rate*BufRateScale.kr(bufnum);
    frames = BufFrames.kr(bufnum);
    duration = frames*(end-start)/rate.abs/s.sampleRate*loops;

    // envelope to clamp looping
    env=EnvGen.ar(
        Env.new(
            levels: [0,1,1,0],
            times: [0,duration-0.05,0.05],
        ),
        gate:t_trig,
    );

    pos=Phasor.ar(
        trig:aOrB,
        rate:rate,
        start:(((rate>0)*startA)+((rate<0)*endA))*frames,
        end:(((rate>0)*endA)+((rate<0)*startA))*frames,
        resetPos:(((rate>0)*startA)+((rate<0)*endA))*frames,
    );
    snd=BufRd.ar(
        numChannels:2,
        bufnum:bufnum,
        phase:pos,
        interpolation:4,
    );

    // add a second reader
    pos2=Phasor.ar(
        trig:(1-aOrB),
        rate:rate,
        start:(((rate>0)*startB)+((rate<0)*endB))*frames,
        end:(((rate>0)*endB)+((rate<0)*startB))*frames,
        resetPos:(((rate>0)*startB)+((rate<0)*endB))*frames,
    );
    snd2=BufRd.ar(
        numChannels:2,
        bufnum:bufnum,
        phase:pos2,
        interpolation:4,
    );

    Out.ar(out,(crossfade*snd)+((1-crossfade)*snd2) * env)
}).play(s, [\out, 0, \bufnum, b]);
)

And we can try this out on the problematic loop from before:

x.set(\t_trig,1,\start,0.2,\end,0.6,\loops,1,\rate,1)

Even though it is triggered a bunch of times, it sounds smooth! This is because, underneath it all, its switching back and forth between loops while crossfading them simultaneously.

8. COOL, but now what?

Now you can branch off with this SynthDef and do all sorts of things. You can add effects to it (maybe I can add more about this). One thing I like to do is that you can immediately do “onset detection”, gather splices, and pattern them!

Here’s a synthdef I cobbled together to determine splicing from arbitrary samples:

(
SynthDef.removeAt("OnsetDetection");

o = OSCFunc({ arg msg, time;
	[time, msg].postln;
	if (msg[3]-~pos1.last>(0.15*s.sampleRate/b.numFrames),{
		"added".postln;
		~pos1.add(msg[3]);
	},{});
},'/tr', s.addr);

SynthDef("OnsetDetection", {
    arg bufnum, out, threshold;
    var sig, chain, onsets, pips, pos, env, speedup=1;

    env=EnvGen.ar(
        Env.new(
            levels: [0,1,1,0],
            times: [0,BufDur.kr(bufnum)/speedup,0],
            curve:\sine,
        ),
        doneAction: Done.freeSelf
    );

    pos=Phasor.ar(0, BufRateScale.kr(bufnum)*speedup, 0, BufFrames.kr(bufnum),0);
    sig = BufRd.ar(2, bufnum, pos,0);

    chain = FFT(LocalBuf(512), sig[0]+sig[1]);

    onsets = Onsets.kr(chain, threshold, \rcomplex,mingap:160);

    // You'll hear percussive "ticks" whenever an onset is detected
    pips = WhiteNoise.ar(EnvGen.kr(Env.perc(0.001, 0.1, 0.2), onsets));
    SendTrig.kr(onsets,0,Clip.kr((pos-300)/BufFrames.kr(bufnum),0,1));
    Out.ar(out,Pan2.ar(sig, -0.75, 0.2) + Pan2.ar(pips, 0.75, 1));
}).add;
)

Once defined, you can run it with the following:

(
~pos1=List.new();
~pos1.add(0);
b=Buffer.read(s,thisProcess.nowExecutingPath.dirname++"/sounds/pianochords.wav",
    action:{
        Synth("OnsetDetection",[\out,0,\bufnum,b.bufnum,\threshold,2.0]); // CHANGE THRESHOLD TO SOMETHING THAT WORKS
});
)

Make sure you change the threshold (above its 2.0) to something where you hear a “click” on the appropriate onset. For example, using the piano from above:

You can hear it automatically detect onsets. And now you can playback individual splices:

f = {arg i;x.set(\t_trig,1,\start,~pos1[i],\end,~pos1[i+1]);}
f.value(0); // plays splice "0"
f.value(1); // plays splice "1"

Or visualize them:

g = {arg i; b.loadToFloatArray(~pos1[i]*b.numFrames,(~pos1[i+1]-~pos1[i])*b.numFrames,{arg array; a=array; {~temp=a.plot;~temp.setProperties(\gridOnX,false,\gridOnY,false)}.defer; })};
g.value(0);
g.value(1); // visualize it

Or sequence them:

(
~player=[1,0,2,0,2,0,0,0,3,3,3,3,4,0,3,0];
t = Task({
    inf.do({ arg i;
        var toPlay;
        0.125.wait;
        toPlay = ~player[i%~player.size].postln;
        if (toPlay>0,{
            toPlay.postln;
            x.set(\t_trig,1,\start,~pos1[toPlay-1],\end,~pos1[toPlay],\loops,1);
        },{});
    });
}).play;
)

Plus much more…but you’ve had enough right?

The vulnerabilities were discovered and disclosed by Aaron Kaiser of the RedRocket CTF team. What Aaron showed me was that on croc (version 8) an exploit could be created to produce a rogue receiver that could backdoor and then bruteforce the SPAKE2 algorithm and receive a file. This attack, though serious, is not covert, as the sender would notice their file being transferred to someplace other than the true receiver. The technical details of this attack are very interesting, thoroughly documented, and cleverly executed and can be found in RedRocket’s blog post about croc’s security flaws.

Luckily, this critical vulnerability seems to have not been exploited. Users that utilized the public relay could have been exploited, but they would have been made aware immediately by recognizing that the sender transferred a file while the receiver never received a file. So far, I have no reports of any users experiencing this behavior. (If you have experienced this behavior, please let me know ASAP via keybase). Also, fortunately, the exploit was found and raised to me privately so I don’t believe it was ever used in nefarious ways. To my knowledge, the exploit found would not work on private relays since it requires to have the rogue recipient first bind itself to the relay (which it cannot do if a relay is set to private).

RedRocket wrote a detailed analysis of croc and outlined several points of failure within it, including this most serious exploit mentioned above as well as a few others. This post will address these vulnerabilities and the changes I made to croc to solve them. These changes are reflected in croc, version 9.

Fixing vulnerabilities and releasing croc version 9

The following is a point-by-point discussion of the flaws discovered in RedRocket’s blog post and the fixes I developed to patch them. To start, though, I want to acknowledge that I am grateful to RedRocket for providing these security observations to me in such a precise and helpful way, and would urge anyone out there to support RedRocket and their CVE discoveries of open-source software. As pointed out by RedRocket:

A small bug can go a long way. It is difficult to implement crypto protocols correctly, especially when a small detail can completely break all security. Unfortunately that’s the case for most protocols. Cyptography is still scary.

I believe this to be very true, and I try to work hard that I can make sure crypto protocols work well in croc so that it remains secure.

Vulnerability in SPAKE2 implementation

As described in the RFC for SPAKE2 the public identities for both parties should be hardcoded by the application. These values should be chosen in a way where the developer can prove that it is unlikely for him to know the discrete logarithm of these values as well.

This has been fixed in pake.go for croc’s SPAKE2 implementation.

As RedRocket notes, my implementation of SPAKE2 did not enforce a specific value for the points along the elliptic curve. In my implementation these points were chosen at random. However, what Aaron of RedRocket cleverly figured out, was that they could choose entities for the curves with known discrete logarithms so that the produced key was much weaker (and breakable through bruteforce).

As suggested by RedRocket, and as described in the forthcoming RFC for SPAKE2, the public identifies should be hardcoded and the values should be chosen in a way that I can prove its unlikely I know the discrete algorithm. So, with help from Travis Scholl, these public identities were generated using a hash of a phrase croc1 and croc2 to prove I don’t know the discrete algorithm. Here is the code to verify the points: SageMath code to generate points.

SIEC use

Unfortunately, croc uses a very unoptimized implementation of an uncommon cofactor-one curve called siec. So for our exploit to work in a reasonable time, we either had to properly implement the curve ourselves or throw more computational power at it. Out of convinience we took the latter option and spun up 10 AWS instances as bruteforce workers.

croc by default uses a less known curve called SIEC. It belongs to a class of curves called “super-isolated” and were first introduced by Travis Scholl. Being super-isolated means it is more difficult to transfer the discrete log problem (the basis of security for classical elliptic curve cryptosystems) to another curve. Hence the security of a SIEC curve does not reduce to the security of the curves around it. These curves necessarily have a small conductor (similar to the curve Bitcoin uses Secp256k1) , and usually admit low degree endomorphisms which could be used to speed up point addition using the well-known GLV/GLS method. Some believe curves with “special features” are less secure. But there are cases where more general objects are less secure, like for genus 3 curves. Because SIEC has not had the popularity of other curves, it has not received the same careful hand optimizing other curves had like secp256k1, P-256, and ec25519. But there is no reason that a similar effort could be made to significantly speed up curve operations. And for those who would rather use a standard curve, it is a minor change to make configurable.

Anyways, the RedRocket team says “unfortunately” here which I take to mean that it was unfortunate for them to have to spin up AWS computers since the siec implementation is less optimized than others - not that the curve itself is unfortunate for cryptography (which I believe is not, as outlined above).

Sender and recipient should not exchange bcrypt hash of key

The sender then answers with its public parameter YYY as well as HkbH_{kb}Hkb​, which is a bcrypt hash of the exchanged key. This is an absolutely terrible idea and already enough to brute force the exchanged key. Unfortunately, bcrypt is used for hashing, which is computationally expensive. With our resources we would need at least a few days to brute force the the passcode based on this hash….Also there should be a check implemented which ensures that Hka and Hkb are different from each other.

This has been fixed to remove any hash exchange of the secure key.

Part of the attack described above was made possible because of a superfluous check that the secure passphrase generated from SPAKE2 are correct between the two parties. I had the same hash for each party which enabled a nefarious party to skip this check by sending back the hash it received from the other.

However, since I’m using AES-GCM with the generated key, I don’t need this check at all and it has been removed from the code. If the two parties have different keys it will be obvious when it fails the decryption, and AES-GCM is able to both decrypt and authenticate the message. This also has the beneficial byproduct of being faster than the previous implementation since bcrypt is not needed anymore.

Room leaks first word of the passcode

The room should not be derived from the passcode but rather uses a separate identifier.

This has been fixed to add a “pin” that designates room so that the weak key is never used as part of the room for exchange.

All generated code-phrases are preceded by a 4-digit pin number that is used to identify the “room” for the transfer to take place. The rest of the code-phrase is used for the shared secret to generate the secure key. This will ensure there is no information about the secure passphrase generation passed between the two parties.

Pathtraversal issues

The sender should not be able to control the path of the file. The recipient should just take the filename from the sender and should save the file to the current working directory.

This has been fixed to prompt overwrites.

In every version of croc the sender is only able to control the path of the file within the recipient’s current working directory. However, the ability to be informed of overwriting files is useful and I have now added it.

To be clear, there are two modes of file transmission in croc:

1. Sender sends a single file. In this case, the recipient will save the file to its current directory no matter where the file originated from the sender. This has been the default behavior for all versions of croc.
2. Sender sends a folder. In this case, the sender cannot send a folder that is not in the sender’s working directory. The recipient will receive the folder and save it and everything in it into the recipient’s working directory.

Previous versions of croc would automatically overwrite files if they were not the same (but only ever in the working directory of the recipient). I changed the default behavior to prompt for overwriting, which can be turned off with -overwrite. The limitation to receive files in the current working directory is not alterable.

Protocol sequence is not enforced

All messages between the recipient and sender are handelt by the processMessage function in croc.go. This function does not enforce the sequence of the messages. If an encryption key is set incoming messages are decrypted. Otherwise the messages are processed without decryption. This allows an attacker to start the protocol with a external ip message and omit the key exchange to use the protocol without encryption. The only thing preventing the sender from sending the file unencrypted is the fact that the go crypto library will throw an error while encrypting data with a nil key and that will result in a panic.

This has been fixed.

In croc version 8, only the PAKE communication is unencrypted and that all other communication requires encryption. So in the now version 9, I have fixed it so that only PAKE communication is explicitly allowed to be unencrypted, while all other messages must be encrypted or it will be rejected with error.

Send-to-relay security

This sender-to-relay connection is “secured” with a SPAKE2 key exchange. Strangely this connection uses the hardcoded passcode pass123 and therefore does not provide security against an active attacker. Although somewhat nasty, we won’t focus on this bug here because confidentiality and authenticity are not crucial for this step.

In this case, the pass123 is simplistic because it is a public relay and it only exchanges the name of the room during the transmissions with the ephemeral key generated with this simple passphrase to encrypt “over-the-wire”. After talking with RedRocket I’ve realized that better security would be to additionally authenticate the public server by generating a keypair and binding the default croc client with the public key so that the public relay (and private) can be authenticated and not just encrypted via a common passphrase. This is work in progress, soon to be released in v10 (probably next week).

This is the traditional way of making showing an audio file in HTML:

1<audio src="drone5.wav" type="audio/wav" controls></audio> 
2<audio src="drone4.wav" type="audio/wav" controls></audio> 

which looks like this:

It turns out its really easy to now get a waveform visualized playback with a drop-in replacement for the audio tag. With about 50 lines of code (below) you can convert audio elements into something much nicer and still practical.

Now the same two audio tags will instead look like this:

How does it work

I simpy wrote a little javascript as a wrapper around wavesurfer.js. The wrapper looks for all the audio selectors and converts them to waveforms. Then it adds SVG buttons for play/pause from feathericons.com.

Here is the entire HTML you need for a drop-in replacement to convert audio tags into nice playable waveforms:

 1<script src="/js/wavesurfer.js"></script>
 2<script>
 3document.addEventListener('DOMContentLoaded', function() {
 4    var els = document.querySelectorAll("audio");
 5    for (var i = 0; i < els.length; i++) {
 6        console.log(els[i].src);
 7        let i_ = i;
 8        let src_ = els[i].src;
 9        let newNode = document.createElement("div")
10        newNode.innerHTML = `<table style="padding-top:16px;padding-bottom:16px;"><tr><td style="position: relative;top: 4px;width:40px;"><div class="controls" style=""> <button class="audiobtn" data-action="play" style="background: #fff; border: none; padding-left:0;padding-right:0;"> <svg class="playicon" xmlns="http://www.w3.org/2000/svg" width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1" stroke-linecap="round" stroke-linejoin="round" class="feather feather-play-circle"><circle cx="12" cy="12" r="10"></circle><polygon points="10 8 16 12 10 16 10 8"></polygon></svg> <svg class="pauseicon" style="display:none;" xmlns="http://www.w3.org/2000/svg" width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1" stroke-linecap="round" stroke-linejoin="round" class="feather feather-pause-circle"><circle cx="12" cy="12" r="10"></circle><line x1="10" y1="15" x2="10" y2="9"></line><line x1="14" y1="15" x2="14" y2="9"></line></svg> </button></div></td><td width="100%"><div class="wave"></div></td></tr></table>`;
11        els[i_].parentNode.insertBefore(newNode, els[i_])
12        els[i].remove();
13        let wavesurfer = WaveSurfer.create({
14            container: document.getElementsByClassName("wave")[i_],
15            waveColor: '#909090',
16            progressColor: '#443e3c',
17            cursorColor: '#ffffff',
18            backend: 'MediaElement',
19            mediaControls: false,
20            hideScrollbar: true,
21            minPxPerSec: 120,
22            normalize: true,
23            height: 64,
24        });
25        wavesurfer.once('ready', function() {
26            console.log('Using wavesurfer.js ' + WaveSurfer.VERSION);
27        });
28        wavesurfer.on('error', function(e) {
29            console.warn(e);
30        });
31        wavesurfer.on('play', function(e) {
32            document.getElementsByClassName("playicon")[i_].style.display = "none";
33            document.getElementsByClassName("pauseicon")[i_].style.display = "block";
34        });
35        wavesurfer.on('pause', function(e) {
36            document.getElementsByClassName("playicon")[i_].style.display = "block";
37            document.getElementsByClassName("pauseicon")[i_].style.display = "none";
38        });
39        newNode.querySelector('[data-action="play"]')
40            .addEventListener('click', wavesurfer.playPause.bind(wavesurfer));
41        wavesurfer.load(src_);
42    }
43});
44</script>

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 02/25/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 01/27/2021.

Declined by The New Yorker on 01/27/2021.

“I warn you, Icarus, fly a middle course: Don’t go too low, or water will weigh the wings down; Don’t go too high, or the sun’s fire will burn them. Keep to the middle way. - Daedalus

for awhile I’ve been attempting to make a synth emulating the korg monotron delay, one of my favorite synths. I’m using the monotron on my current album a lot and wanted to make a softsynth of it which would be a little easier to use in my other scripts. this is a synth for norns that is close to what I envisioned, with some cool features not present in the monotron (pitch stability) but also still missing some of the intense time-swooping (I hope to get there soon).

the synth is presented on the norns with three controls shown in the landscape. the sun shows feedback and levels, the water shows filter and the position shows time. i recommend interacting with these facets in realtime to contour the sound. like icarus, you can keep to the middle, you can get burned up by the sun, or you can get swallowed up by the sea.

requirements

• norns
• midi keyboard

documentation

(plug in midi keyboard first)

• E1 = time
• E2 = filter
• E3 = feedback
• K1/2/3 does a bounce on those three

moving time forward causes the sun to more easily collapse.

lots of parameters in the menu.

Install

;install https://github.com/schollz/icarus

plonk (/plɒŋk/) - to play a musical instrument, usually not very well but often loudly - Cambride Dictionary

plonky is a keyboard and sequencer. i made it to be able to play mx.samples directly from the grid. the grid layout (and name) is inspired by the plinky synth, i.e. it is a 8x8 layout with notes spaced out between columns by a specified interval (default is C-major scale spaced out by fifths).

Requirements

• norns
• grid (optional)

Documentation

use the grid to play an engine. by default the engine is “PolyPerc”, but if you install mx.samples you can also play that by switching “PLONKY > engine" via parameters.

each 8x8 section of the grid is a voice (1 voice for 64-grid, 2 voices for 128-grid). you can play notes in that voice by pressing pads. the notes correspond to a C-major scale, where each column is a fifth apart (all adjustable in menu).

arps: you can do arps by turning E2 or E3 to the right. in “arp” mode you can press multiple keys and have them play. in “arp+latch” mode the last keys you pressed will play. change the speed using the “PLONKY > division” parameter in the menu.

patterns: you can record patterns by pressing K1+K2 (for voice 2 press K1+K3). press a note (or multiple) and it will become a new step in the pattern. you can hold out a step by holding the notes and pressing K2 (for voice 2 press K3). you can add a rest by releasing notes and pressing K2 (for voice 2 press K3). when done recording press K1+K2 (for voice 2 press K3). to play a pattern press K2 (for voice 2 press K3).

midi keyboard support: every voice now has a “midi-in” which you can assign to it a midi keyboard for input. the midi-in keyboards only work if the voice is selected (i.e. only two work at a time). note: its important to note that the midi keyboard support is basically allowing to use a midi keyboard as a input into plonky - emphasis on plonky. meaning, plonky doesn’t accept notes from a midi keyboard that aren’t in the plonky grid area (even if there is no grid attached). so if you have a C major scale plugged in, plonky will only respond to the white keys on your keyboard. also, a grid only lets you play two voices at once, so similarly plonky will only respond to two midi keyboards at the same time (you can switch voices to switch which keyboards are used if they are assigned across each voice). I realize these are weird limitations, but its the only way I can see forward to keep the grid-based plonky and the now keyboard-based plonky in unison.

Install

https://github.com/schollz/plonky

from maiden:

;install https://github.com/schollz/plonky

granchild is a granular grandchild. this script was born out of and inspired by @cfd90’s clever twine and @justmat’s inspiring mangl, both themselves based on @artfwo’s amazing glut script, which itself was inspired by @kasperskov’s grainfields. i consider this script to be a grandchild of glut, and child of the other two - using some ideas from twine (randomization in parameters) and some code from mangl (lfos, greyhole integration) and the engine of glut, to make a granular sequencer to fit my personal musical journey.

their are a lot of granulation scripts based on glut, here’s what’s different about granchild:

• “jitter” and “spread” of granulation always oscillate, with random frequencies
• “size” and “density” of granulation are quantized, and easily accessed (see layout)
• sequencer is quantized (using @tyleretters’s lattice)
• voices limited to only 4
• samples mapped to 18 keys
• can record live input
• granulation can utilize stereo samples
• each voice has two scenes that can be toggled between

Requirements

• norns
• grid optional

Documentation

here’s how the buttons are laid out. use e2 and e3 on the norns to move around and k2/k3 to toggle things. the grid just mirrors the norns screen, so this script works just fine without a grid. grid 64 users: hold k1 to switch between the the first and second pair of voices.

note: when you use the record button for live input, all recordings are automatically saved permanently in ~/dust/audio/granchild.

Old versions

see github

Install

from maiden:

;install https://github.com/schollz/granchild

us as a keyboard instrument or in another script. mx.samples provides an accessible way to utilize the vast trove of free instrument sample libraries. for instance, you can load in a piano that has been sampled on multiple dynamics and variations and key releases.

the core of the script + supercollider engine handles the voices and the instruments. you can have unlimited samples on disk because samples are loaded dynamically - only loading into memory when needed (max 200 samples can be loaded though). no latency on loading because it will load in the background (and use the closest pitched sample in the meantime).

my motivation for this script was to have a really nice sounding piano on the norns. also i wanted to be able to have a library of samples that works well with tmi (see studies/study1.lua and demo above) but can also be used for future projects in the idea galaxy.

a massive thank you to @zebra for helping me post-process the UofI piano samples - there is a great script available to “de-pop” samples that had glitches in recording, thanks to that work. also this project wouldn’t be possible without the generosity of the folks submitting their samples to be used freely at pianobook.co.uk and at the University of Iowa Electronic Music department.

requirements

• norns
• midi controller OR use in a script

documentation

mx.samples can be used as a keyboard (selecting sound+parameters via menus) or as a library (selecting sound+parameters via code).

as a keyboard

you can use mx.samples as a keyboard. just plugin your midi keyboard, open the script and choose a sample. samples are available to download (processed by me, you can process your own too - see below).

there are a bunch of effects (filters / delay) and options (tuning, down-sampling, playing releases, velocity scaling) available from the parameters menu PARAMETERS -> MX.SAMPLES.

“warming” up the keyboard: the very first note you play will not “play” (known bug) because it is loading the sample. every subsequent new note will re-pitch the loaded sample or it will load in a sample for that note to be used the next time (so no latency from load). this means that you can get the best sound by playing the notes you want to play once before you play them.

as a library

you can use mx.samples as a lua library for your project (see demo above). the api is pretty simple to include this into another script if you want a bunch of different voices. (another goal here - to use with tmi). instruments are loaded dynamically so you can add as many as you want.

see study1.lua for an example that uses tmi to do the sequencing.

basically the syntax would be:

 1mxsamples=include("mx.samples/lib/mx.samples")
 2engine.name="MxSamples"
 3skeys=mxsamples:new()
 4
 5-- play an instrument
 6skeys:on({name="ghost piano",midi=60,velocity=120})
 7skeys:on({name="box violin",midi=42,velocity=120})
 8
 9-- turn them off
10skeys:off({name="ghost piano",midi=60})
11skeys:off({name="box violin",midi=42})

the parameters can be added into the on function as well:

1skeys:on({
2	name="ghost piano",midi=60,velocity=120,
3	attack=1,
4	release=3,
5	-- every parameter in the menu is available here
6})

getting samples

this script will allow you to download samples that i’ve already processed. in theory you can use any kontakt / vst sample pack if you have access to the raw audio. in the samples/ folder there is a utility script convert.py that is specific to each type of sample and shows basically how its done (its easy). here’s an example for the UofI piano.

the current samples are from the following sources, which are free and do not restrict to distributing them for this purpose:

• The University of Iowa Musical Instrument Samples database which “may be downloaded and used for any projects, without restrictions”.
• the pianobook which states that “There are NO restrictions on their use (except selling them on as your own samples)”

Install

;install https://github.com/schollz/mx.samples

What does this tiny hack do?

Here’s a demo of the tiny hack! Basically this hack allows me to easily play the Monotron as drone while playing other synthesizers. In this demo I’m having the Monotron drone on a low C, while I play chords on top of it!

Why do this tiny hack?

The Korg Monotron usually don’t turn “on” until you press the ribbon keyboard. So if you want the Monotron to drone on one note you’d have to press your finger down on it and leave your finger there. Using this hack you can replace your finger with a potentiometer which always keeps it “on” and also sets the pitch to whatever you want! Its still manually controlled, but now using a pot instead of a ribbon.

How do I do this tiny hack?

If you already have some diodes and resistors and a potentiometer, this hack will cost you nothing. Otherwise you can get those basic utilities:

• Korg Monotron (~$55)
• 10k potentiometer
• 10k resistor
• Diodes
• Breadboard
• Soldering iron

To connect it you will first need to solder wires to the back of the Korg Monotron. Unscrew the Korg Monotron and attach three wires to the pads labeled “VCC”, “GND”, and “GATE” (yes “PITCH” is a pad, but we will not use it).

Then you can build a simple circuit using the resistor, potentiometer and the diodes.

Why does it work? Basically the diodes are there to drop the voltage of Monotron (which runs > 4V) to the highest voltage of a pitch (~3 V). Then the potentiometer and resistor form a voltage divider that divides the voltage even further proportional to turning it. This way the knob turns between allowable notes (low notes to the left and high notes to the right).

Uh oh something bad happened!

Oh no! I’m happy to help - just DM me on Twitter (@yakczar) or email me.

SuperCollider is a free platform for audio synthesis and algorithm composition. It really excels at real-time analysis, synthesis and processing and provides a framework for seamless combination of audio techniques like additive and subtractive synthesis. The following is a tutorial to learn how to make a drone in SuperCollider (e.g. Sunn O))))¹)

Before you begin

Before starting, make sure you have SuperCollider and plugins (if you want).

• Install SuperCollider
• Install SuperCollider plugins (optional, but recommended)

If you have these basic things installed, you are good to go. Feel free to do a SuperCollider tutorial if you want, but I’ll assume you don’t know how to use it.

Five steps to get in the drone zone

We are going to make the drone I call “Dreamcrusher”, a primal sawtooth drone. We’ll get there from a simple sine wave, in just 5 simple steps.

Dreamcrusher (final version):

1. Tone

We’ll start with a basic tone and turn it into a drone. First open SuperCollider and “boot the server” by selecting Server -> Boot Server from the menu.

Now paste the following code as the starting drone. this drone is simply a C3 sine wave (C2 = 131.81 hz).

Dreamcrusher (version 1)

Code (version 1)

(
x = SynthDef("basic",
    {
        arg hz=131.81, amp=0.5;
        var sig; // define variables we need
        
        // the drone zone!
        sig = SinOsc.ar(
            freq:hz,
            mul:amp,
        );
        
        // make two versions, one for left and one for right
        sig = sig.dup;
        
        // make sound!
        Out.ar(0,sig);
    }
).play;
)

To run this code, click your cursor anywhere in the code that you just pasted and press Ctl+<enter>. This runs all the code between the first and last parentheses. You should hear a very low sound (maybe use headphones to hear it!).

To stop it just press Ctl+<period>.

Another cool trick is that you can change things while its playing, for instance we can change the frequency and volume (use Ctl+Enter to run each line).

x.set("hz",65.41*2); // up can octave
x.set("amp",0.01); // volume down

The reason this works is that we defined the drone into the x variable and defined arguments (see the line “arg hz=65.41, amp=0.5;”).

2. Add overtones

Mow lets make it less boring with overtones. Overtones are multiples of fundamental at lower volumes. We can easily just repeat what we wrote before but wrap it inside a Mix.ar() function in an array (denoted by the brackets).

Dreamcrusher (version 2)

Code (version 2)

(
x = SynthDef("basic_w_overtones",
    {
        arg hz=131.81, amp=0.5;
        var sig;

        // the drone zone!
        sig = Mix.ar([
            SinOsc.ar(
                freq: hz,
                mul: amp,
            ),
            SinOsc.ar(
                freq: hz*2,
                mul: amp/2,
            ),
            SinOsc.ar(
                freq: hz*4,
                mul: amp/4,
            )
        ]);

        // spread the signal
        sig = Splay.ar(sig);

        // make sound!
        Out.ar(0,sig);
    }
).play;
)

I made sure to decrease the volume of each increasing frequency using by dividing the amp by a multiple of 2.

3. Different oscillators!

Sine waves are nice, but there are lots of different oscillators we can use. we can swap out SinOsc.ar (sine wave) for VarSaw (sawtooth), SinOscFB (sine w/ feedback), Pulse, or many others. Here I’ve opted for the saw wave:

Dreamcrusher (version 3)

Code (version 3)

(
x = SynthDef("basic_w_overtones_varsaw",
    {
        arg hz=131.81, amp=0.5;
        var sig;

        // the drone zone!
        sig = Mix.ar([
            VarSaw.ar(
                freq: hz,
                mul: amp,
            ),
            VarSaw.ar(
                freq: hz*2,
                mul: amp/2,
            ),
            VarSaw.ar(
                freq: hz*4,
                mul: amp/4,
            )
        ]);

        // spread the signal
        sig = Splay.ar(sig);

        // make sound!
        Out.ar(0,sig);
    }
).play;
)

Try other oscillators! See what happens.

4. Add modulation

Modulation will bring in organic wavering to the drone. There are many ways to do this, but my favorite is to use a randomly oscillating sine wave with a long period.

We can get randomness all sorts of different ways to get modulation.

Stepped randomness

Using the LFnoise0.kr:

{LFNoise0.kr(freq:10)}.plot(1)

Oscillating randomness

Then we can make a sine wave which uses that stepped noise as its frequency:

{SinOsc.kr(freq:LFNoise0.kr(freq:2)*4)}.plot(4)

The sine wave naturally has values between -1 and 1, so we might need to change that. its easy to change to any range you want with the .range(min,max) function (or by using LinLin.kr). Here i can change it to 0-100:

{SinOsc.kr(freq:LFNoise0.kr(freq:1)).range(0,100)}.plot(4)

Drunken walk

You can dream up any sort of modulation that you like. another of my favorite modulation sources of is a nice “drunken walk” (best to replace both 5’s by the number you want):

{VarLag.kr(LFNoise0.kr(5).range(0,100),1/5,warp:\sine)}.plot(4)

You can easily swap in and out these modulations, just make sure to convert the .range(min,max) to the range you need it.

Now throw those into the drone to modulate the frequencies and the width (a parameter associated with the VarSaw ) of our oscillator:

Dreamcrusher (version 4)

Code (version 4)

(
x = SynthDef("basic_w_overtones_varsaw_modulation",
    {
        arg hz=131.81, amp=0.5;
        var sig;
        
        // the drone zone!
        sig = Mix.ar(
            VarSaw.ar(
                freq: Lag.kr(hz * SinOsc.kr(LFNoise0.kr(1)).range(0.99,1.01),1),
                width: SinOsc.kr(LFNoise0.kr(1)).range(0.4,0.6),
                mul: amp ,
            ) +
            VarSaw.ar(
                freq: Lag.kr(2*hz * SinOsc.kr(LFNoise0.kr(1)).range(0.99,1.01),1),
                width: SinOsc.kr(LFNoise0.kr(1)).range(0.4,0.6),
                mul: amp/2,
            ) +
            VarSaw.ar(
                freq: Lag.kr(4*hz * SinOsc.kr(LFNoise0.kr(1)).range(0.99,1.01),1),
                width: SinOsc.kr(LFNoise0.kr(1)).range(0.4,0.6),
                mul: amp/4,
            )
        );
        
        // spread the signal
        sig = Splay.ar(sig);
        
        // pan
        sig = Balance2.ar(sig[0] ,sig[1],SinOsc.kr(
            LFNoise0.kr(0.1).range(0.05,0.2)
        )*0.1);
        
        // make sound!
        Out.ar(0,sig);
    }
).play;
)