This project started with the slightly unreasonable idea of building a stringed instrument that could teach itself how to play. The result, so far, is the Gippsland Dulcimator: a home-built four-string electric dulcimer-style instrument using steppers, servos, ESP32s and now a Raspberry Pi 4 as a teaching brain.
The Gippsland Dulcimator — Building a Self-Playing Electric Dulcimer from Scratch
This project started with the slightly unreasonable idea of building a stringed instrument that could eventually teach itself how to play.
The result, so far, is the Gippsland Dulcimator — “Dulcie” for short — a home-built six-string electric dulcimer-style instrument using steppers, servos, ESP32s and now a Raspberry Pi 4 as a teaching brain.
It is not intended to imitate a guitar or a conventional mountain dulcimer exactly. The idea is to build an instrument around what motors and electronics can do well, rather than force the machine to behave exactly like a human player.
The instrument
The six strings are arranged as a south melody pair, two centre drone strings, and a north melody pair.
For early development, one string from each melody pair is parked sideways off the board. That leaves four active strings: one south melody string, the two centre drone strings, and one north melody string.
The player stands on the south/front side of the instrument.
The instrument mounts on the outside of the box lid. The electronics live inside the box. The lid hinge is at the north/back side, and the tuners and stepper motors are at the east end.
The two outside melody positions are handled independently, while the two centre strings remain fixed-pitch drones.
One of the useful discoveries during testing was that fret buzz could be dealt with by changing the overall string geometry. The bridge was raised by 3.2 mm, which improved the string clearance and gave the frets enough room to work properly.
That also introduced another useful complication: the strings now rise slightly as they approach the pluckers, so the plungers need to travel farther at some fret positions than at others.
Rather than trying to make the mechanics geometrically perfect, the plan is to let the control system compensate for that in software.
Sliders, plungers and pluckers
Each active melody string pair is controlled by a motor-driven carriage.
The sliders use NEMA 17 stepper motors, GT2 belts and linear bearings. The belts are now fitted and the carriages are assembled.
The string is pushed down behind the selected fret by a small servo-driven plunger mechanism. The plungers use MG90S micro servos operating cams.
A particularly useful bit of workshop discovery was that the actual piece that contacts the string can be made as a removable slip-in/slip-out tip. That means the effective plunger height can be changed without rebuilding the whole mechanism.
The existing tips also appear to show very little wear.
The plucking mechanisms use DS3218 Pro servos. Two plucker assemblies are already fitted and tested, with the remaining mechanisms being completed as the build progresses.
The mechanical side is deliberately adjustable. Clamps, removable tips, bridge height and software-controlled press depth all give room to tune the machine rather than depending on every dimension being perfect first time.
That has become an important design principle of the project:
The mechanics need to be repeatable, not perfect. The software can learn the corrections.
Electronics
The Dulcimator uses several ESP32 WROOM boards.
One ESP32 acts as the Boss controller. It will receive higher-level instructions and coordinate the machine.
The other ESP32s handle local motor and servo duties.
Stepper motors are driven through TMC2209-type drivers.
Servo and logic power are kept separate from the heavier stepper supply where practical, using buck regulators and separate distribution.
The Dulcimator is intended to remain reasonably portable and self-contained, so battery power and compact DC supplies are preferred over running the whole thing from a bench power supply.
Giving Dulcie a brain
The next major stage is the interesting one.
A Raspberry Pi 4 Model B with 2 GB RAM has now been bought to act as the temporary teaching brain.
The Pi will connect to the Boss ESP32 by USB serial.
The planned relationship is:
Pi 4 = teacher and temporary brain
Boss ESP32 = machine foreman
Other ESP32s = local muscle controllers
The Boss does not need to understand music. It only needs to understand physical instructions such as:
HOME
MOVE
PRESS
RELEASE
PLUCK
STOP
The Pi does the thinking.
Dulcie will listen to herself
The instrument has a humbucker pickup and is played through an old Fender Champ 15 amplifier.
The amp has a headphone output, and the plan is to feed that signal into the Raspberry Pi through a small USB audio-input dongle.
That means the Pi will hear the actual electrical output of the instrument rather than relying on a microphone in the shed.
The Pi does not need stored recordings of musical notes to compare against.
A musical note has a known fundamental frequency. For example:
A3 = 220 Hz
If the Pi asks Dulcie to play A3 and measures 218 Hz, it knows the result is flat. If it measures 222 Hz, it knows it is sharp.
The first pitch-detection approach will probably use autocorrelation or YIN, rather than something crude like counting zero crossings, because a plucked string contains harmonics and changes as it decays.
Early lessons will also keep the problem simple by isolating one melody string at a time and damping other strings if sympathetic vibration becomes a problem.
The teaching idea
The core idea behind the project has now become:
Try → Listen → Measure → Adjust → Remember
For example, the Pi might tell the Boss:
Move the melody slider to a trial position.
Press the string.
Wait for the mechanism to settle.
Pluck.
The Pi then listens. It measures the pitch and compares it with the target. If the note is flat, move slightly one way. If it is sharp, move slightly the other way. Try again.
Once a setting consistently produces the correct note, the Pi stores it. That remembered information can include far more than just pitch — for each note Dulcie can eventually remember the slider position, required plunger depth, pluck setting, movement time, relative volume, sustain and repeatability.
The next time the note is required, the machine starts from the known-good settings rather than guessing again.
This is not machine learning in the glamorous AI sense. It is much simpler and, for this job, probably much more useful. The machine tries something, measures the result and remembers what worked.
Why the Pi 4?
There was some temptation to jump straight to something much more powerful such as an NVIDIA Orin.
The conclusion was: don’t.
A Pi 4 with 2 GB is more than capable of serial control, audio capture, pitch detection, timing, logging and storing learned settings. If the Dulcimator later genuinely needs vision, neural-network audio processing or some other heavy AI job, then a more powerful computer can be justified.
For now the rule is: prove the need before buying the horsepower.
Where it is now
The project has moved a long way from sketches and loose parts.
The frets work. The bridge geometry has been corrected. The belts are fitted. The plungers are mechanically adjustable. The plucker system has been tested. The ESP32 control architecture is established. The Pi 4 has arrived.
The next immediate jobs are very small and deliberate: fit the Pi with a microSD card, add the USB sound-input hardware, get the Pi running, and prove that it can hear and measure one Dulcimator note correctly.
After that comes the first proper lesson. Not a tune. Not a scale. Just one note.
If Dulcie can be told to attempt one note, listen to herself, correct the mistake and remember what worked, then the basic idea behind the whole machine has been proved.
Early lessons will keep the problem simple by working with one active melody string at a time and damping the other active strings if sympathetic vibration becomes a problem.
And that is where the project is now.
