[DECtalk] The Quest to Save Stephen Hawking's Voice

[Brandon Tyson]

Hi,

Wow, I'm amazed.
The fact that you can take a voice from 30+ years ago and allow it to
work on modern technology, so that he can still have his same voice
and be himself, is really fascinating to me.

Thanks for sharing,

B

On 3/19/18, Josh Kennedy <joshknnd1982 at gmail.com> wrote:
> Yes, keynote is older than eloquence. And infovox230 is about as old as
> decTalk maybe a bit older.
>
>
> Sent from Mail for Windows 10
>
> From: Damien Garwood
> Sent: Monday, March 19, 2018 04:45
> To: DECtalk
> Subject: Re: [DECtalk] The Quest to Save Stephen Hawking's Voice
>
> Hi,
> Tony: My understanding is that a fork is precisely what you’re talking
> about, code, in this case for a speech synth, which has a common ancestor
> but which have branched off into different products, in this case MITalk
> which branched off into DECTalk etc. It’s like Twitter clients today,
> Qwitter has independently morphed into Chicken Nugget, while forks of
> Qwitter, I.E. the Qube and TWBlue have their own quirks and improvements.
> The only time it wouldn’t be a fork is if it were completely rewritten from
> the ground up, which I suppose is also possible (Apollo and Orpheus come to
> mind here). I know a lot of people take “forks” to be simply rebranded
> copies and so the word has somehow harboured negative connotations since,
> but a fork is simply taking a common codebase and branching it out from the
> original, whether you make your own improvements or not.
> Josh. Regarding Eloquence, I said “sounds like”. Obviously having no access
> to either of the codebases I can’t say with 100% certainty that they are
> forks. I’m saying that, from an audible point of view, they sound like they
> are either forks, or at least one was written specifically to have the
> audible qualities of the other. The only difference is that Keynote has very
> expressional pitch changes, and sounds slightly more...Distorted isn’t the
> word I’m looking for, but it’s the closest I can think of right now.
> Eloquence, on the other hand, sounds slightly smoother, but its expression
> is lousy unless you really tweak at the intonation parameters, at which
> point it sounds a bit too enthusiastic.
> Am I right in thinking that Keynote is older than Eloquence? Never managed
> to find the history behind those two synths so don’t even have an idea as to
> what motivated them or how they came to be. I know Eloquence began its life
> as ViaVoice, and that’s all.
> I do remember at one point, when my ears weren’t so tuned into speech
> synthesis, I had this weird belief that Eloquence was DECTalk’s successor. I
> can kind of see why, if you really, and I mean really, tweaked DECTalk, you
> could get it to sound at least slightly like Eloquence, but it does take a
> lot of tweaking and even then it isn’t that accurate. In fact, I’d say,
> while their voices are completely different, I think at least Keynote and
> DECTalk are alike in their delivery of expression and intonation.
> Cheers.
> Damien.
>
> From: Josh Kennedy
> Sent: Monday, March 19, 2018 12:30 AM
> To: DECtalk
> Subject: Re: [DECtalk] The Quest to Save Stephen Hawking's Voice
>
> Really? Eloquence is a fork of keynote? How can you say that?
>
>
> Sent from Mail for Windows 10
>
> From: Tony Morales
> Sent: Sunday, March 18, 2018 19:25
> To: DECtalk
> Subject: Re: [DECtalk] The Quest to Save Stephen Hawking's Voice
>
> They sound similar because they have a common ancestor--MITalk.
>
> 30. The MIT MITalk system, by Jonathan Allen, Sheri Hunnicutt, and Dennis
> Klatt, 1979.
>
> http://www.cs.indiana.edu/rhythmsp/ASA/AUfiles/30.AU
>
> 32. The Speech Plus Inc. `` Prose-2000'' commercial system, 1982.
>
> http://www.cs.indiana.edu/rhythmsp/ASA/AUfiles/32.AU
>
> 33. The Klattalk system, by Dennis Klatt of MIT, which formed the basis for
> Digital Equipment Corporation's DECtalk commercial system, 1983.
>
> http://www.cs.indiana.edu/rhythmsp/ASA/AUfiles/33.AU
>
> Audio samples from Klatt's History of Speech Synthesis.
>
> Thanks,
>
> Tony
>
>
>
> On Mar 18, 2018, at 4:04 PM, Damien Garwood <damien at dcpendleton.plus.com>
> wrote:
>
> Hi,
> You know, hearing the samples in that article, I can't say I would be
> surprised if DECTalk was almost a fork of that synth. Just like it sounds
> like Eloquence is a fork of Keynote, and ESpeak is a fork of Orpheus (if not
> the vocal properties, then at least the dictionary and stress rules).
> Let us hope that this sees a resurgeance of interest on formant synthesis,
> as concat synthesis, which currently seems to be dominating the TTS market,
> just doesn't cut it for screen readers in my opinion.
> As for Stephen Hawking. May he rest in peace, may his spirit be in a happier
> place, may his great work be remembered, may his confidence shine through
> others, may his battles for accessibility continue to be fought, and may his
> voice be preserved forever as a reminder of the progress that was made in
> his lifetime in order to live a rich and fulfilling life despite his
> disabilities.
> Cheers.
> Damien.
> -----Original Message----- From: Jayson Smith
> Sent: Sunday, March 18, 2018 10:04 PM
> To: DECtalk Discussions
> Subject: [DECtalk] The Quest to Save Stephen Hawking's Voice
>
> Here's an interesting article I thought some of you might like.
>
>
> Jayson
>
>
> --------------------
>
>
> The quest to save Stephen Hawking's voice
> How a Silicon Valley team helped rebuild his distinctive robotic sound
> By Jason Fagone
>
> Eric Dorsey, a 62-year-old engineer in Palo Alto, was watching TV
> Tuesday night when he started getting texts that Stephen Hawking had
> died. He turned on the news and saw clips of the famed physicist
> speaking in his iconic android voice — the voice that Dorsey had spent
> so much time as a young man helping to create, and then, much later, to
> save from destruction.
>
> Dorsey and Hawking had first met nearly 30 years earlier to the day. In
> March 1988, Hawking was visiting UC Berkeley during a three-week lecture
> tour.
>
> At 46, Hawking was already famous for his discoveries about quantum
> physics and black holes, but not as famous as he was about to be. His
> best-seller, “A Brief History of Time,” was a week away from release,
> and Californians were curious about this British professor from the
> University of Cambridge, packing the seats of his public talks,
> approaching him at meals. Hawking motored into buildings and onto stages
> in a wheelchair with a seat of maroon sheepskin, zooming around with the
> nudge of a joystick, grinning as he left journalists and his nurses in
> the dust.
>
> When he spoke, it was in the voice of a robot, a voice that emerged from
> a gray box fixed to the back of his chair. The voice synthesizer, a
> commercial product known as the CallText 5010, was a novelty then, not
> yet a part of his identity; he’d begun using it just three years before,
> after the motor neuron disease amyotrophic lateral sclerosis stole his
> ability to speak. Hawking selected bits of text on a video screen by
> moving his cheek, and the CallText turned the text into speech. At the
> start of one lecture, Hawking joked about it: “The only problem,” he
> said, to big laughs, “is that it gives me an American accent.”
>
> Dorsey was with Hawking for part of that trip, tagging along as a sort
> of authority on the voice, explaining its workings to journalists. He
> worked at the Mountain View company that manufactured the CallText 5010,
> a hardware board with two computer chips running custom software.
>
> An upbeat 32-year-old, Dorsey was quiet by nature, but driven. He had
> joined Speech Plus as an intern, attracted by its mission to help the
> voiceless and the disabled; now he led a team of engineers, and at least
> 20,000 lines of his own code were in the CallText, the product that gave
> voice to the most celebrated scientist of his era.
>
> “We are getting close to answering the age-old questions,” Hawking said
> at the close of a lecture. “Why are we here? Where did we come from?
> Thank you for listening to me.”
>
> At the end of his California tour, the physicist gave Dorsey a signed
> copy of his new book, his thumbprint pressed onto the inside cover.
>
> Hawking returned to Cambridge, Dorsey to his life in California.
>
> Twenty-six years went by before they would cross paths again.
>
> In tech years, that is a millennium. The Internet happened. Silicon
> Valley boomed, busted, boomed again. Apple, Amazon, Facebook, Google, Uber.
>
> Dorsey, meanwhile, left Speech Plus, which went bankrupt and was sold to
> a series of other companies. He got married and had kids. He joined a
> Buddhist temple. He eventually left the field of speech technology
> altogether, becoming an engineering leader at DVR maker TiVo.
>
> Tech, he’d learned, moves so fast. “There’s a new iPhone every year,”
> Dorsey says. “Everything just kind of gets buried in the dustbin of
> history very, very quickly.”
>
> That’s why, when an email from Cambridge University arrived out of the
> blue in 2014, Dorsey was surprised. It came from Hawking’s technical
> assistant, Jonathan Wood, who was responsible for Hawking’s
> communications systems.
>
> Wood explained something so improbable that Dorsey had trouble
> understanding at first: Hawking was still using the CallText 5010 speech
> synthesizer, a version last upgraded in 1986. In nearly 30 years, he had
> never switched to newer technology. Hawking liked the voice just the way
> it was, and had stubbornly refused other options. But now the hardware
> was showing wear and tear. If it failed entirely, his distinctive voice
> would be lost to the ages.
>
> The solution, Wood believed, was to replicate the decaying hardware in
> new software, to somehow transplant a 30-year-old voice synthesizer into
> a modern laptop — without changing the sound of the voice. For years, he
> and several colleagues in Cambridge had been exploring different
> approaches. What did Dorsey think?
>
> Thirty years old? He thought. Oh, my God.
>
> It wouldn’t be easy. They might have to locate the old source code. They
> might have to find the original chips and the manuals for those chips.
> They couldn’t buy them anymore, the companies don’t exist. Solving the
> problem might mean mounting an archaeological dig through an antiquated
> era of technology.
>
> But it was for Stephen Hawking.
>
> “Let’s get it done,” Dorsey said.
>
> --------------------------------------------------------------------------------
>
> The poet Longfellow once wrote that the human voice is “the organ of the
> soul.” More than any other part of us, our voice expresses who we are,
> and the smallest fluctuations swing meaning in ways that are hard for
> computers to understand. You speak a sentence, and the intonation rises
> or falls depending on whether you’re making a statement or asking a
> question. You do it without thinking, but a computer has to make a guess.
>
> Today’s synthesized voices, like Apple’s Siri, rely on prerecorded
> libraries of natural sound. Voice actors record huge libraries of words
> and syllables, and software chops them up and reassembles them into
> sentences on the fly. But 30 years ago, computers could only produce a
> “stick-figure version” of a human voice, says Patti Price, a speech
> recognition specialist and linguist in Palo Alto.
>
> Back then, she worked as a postdoc in the Massachusetts Institute of
> Technology lab of Dennis Klatt, a tall, thin, opera-loving scientist
> originally from Wisconsin. Klatt is the godfather of Hawking’s voice. He
> blasted his own throat with X-rays to measure the shape of his voice box
> as he articulated certain sounds, and then developed a software model of
> speech, the Klatt Model, based on his own voice.
>
> Speech Plus took Klatt’s model, improved on it, and commercialized it in
> various products, including the CallText 5010. One of Dorsey’s
> contributions was to write an algorithm that controlled the intonation
> of the voice, the rise and fall of words and sentences. Speech Plus
> would sell thousands of CallText systems, though many customers
> complained that the voice sounded too robotic.
>
> But Hawking liked it.
>
> True, it was robotic, but he appreciated that it was easy to understand:
> “noise-robust,” as Price explains. The shape of its waveform was more
> like a series of plateaus than the steep mountain cliffs of human
> voices, which fall off more sharply. The flattish slope of Hawking’s
> voice made it cut through noise in amphitheaters and lecture halls. He
> often began his speeches with the same line — “Can you hear me?” — and
> the audience would respond with an enthusiastic “Yes!”
>
> “It’s got a ring to it that sticks out,” Price says.
>
> “It’s very intelligible,” Dorsey says. “You can listen to it for a long
> time, and it’s not irritating.”
>
> Hawking’s only complaint was that it didn’t have a British accent.
>
> Over the years, as synthetic voices grew more natural, taking advantage
> of faster chips and cheap storage, Hawking had chances to upgrade. In
> 1996, a Massachusetts speech technology company called Nuance, which had
> acquired the remains of Speech Plus, upgraded the CallText with evolved
> software code that made the voice sound fuller and faster, less robotic,
> with shorter pauses between sentences — to the engineers, an obvious
> improvement.
>
> They sent Hawking a sample of the new voice, thinking he’d be pleased.
> He was not. He said the intonation wasn’t right. He preferred the 1986
> voice, the one modulated by Dorsey’s intonation algorithm. Hawking would
> stick with that one.
>
> “I keep it because I have not heard a voice I like better,” he once
> said, “and because I have identified with it.” He could change to a
> smoother voice, but then he wouldn’t sound like himself.
>
> “To Stephen, his equipment is like a part of his body,” said Wood, his
> chief technical aide. “To upgrade him to a new piece of software or a
> new piece of hardware … he’s having to change a physical part of himself.”
>
> --------------------------------------------------------------------------------
>
> Starting around 2009, Wood and several others at Cambridge began trying
> to separate Hawking’s voice from the failing CallText hardware. The
> group would included Peter Benie, a computer guru at the university;
> Pawel Wozniak, a local engineering student; and Mark Green, an
> experienced electrical engineer with Intel, which had a long
> relationship with Hawking.
>
> One option they considered was tweaking a modern synthetic voice like
> Siri to sound more like Hawking. But Siri-type systems rely on the vast
> computer power of Internet clouds, and Hawking couldn’t be constantly
> tethered to the Internet. Benie also tried a completely different
> approach. He wrote a software emulator for the CallText — essentially a
> program that would fool a modern PC into thinking it was actually the
> old CallText. But the samples it produced didn’t sound faithful enough
> for Hawking’s taste.
>
> By the time Cambridge reached out to Dorsey in 2014, they were
> investigating a third avenue: track down the old CallText source code,
> now owned by Nuance, and port it to Hawking’s laptop, transplanting the
> old voice into a fresh new body.
>
> Was it possible? Dorsey had no idea. It depended on whether he could
> find the source code, or, failing that, information that would let him
> reverse-engineer the source code.
>
> He started emailing colleagues he hadn’t seen in 30 years, asking if
> they had any CallText bric-a-brac still laying around: boards, chips,
> manuals. One guy found an actual CallText board in his garage. Others
> located dusty schematics.
>
> It had the feel of a mad scramble through an earlier era of technology.
> But people everywhere leaped at the chance to help. “The goal is to save
> his voice,” Dorsey said. “Once you go to somebody — ‘I need you to help
> save Stephen Hawking’s voice’ — they immediately wake up.”
>
> His closest collaborator in Palo Alto soon became Price, the speech
> technologist who had once studied with Klatt, the godfather of Hawking’s
> voice. She was a master at analyzing audio samples, comparing one to
> another and using their audio fingerprints to reverse engineer how they
> must have been created.
>
> Dorsey’s archaeological quest for old code turned out to be a
> frustrating one. No one at Nuance was able to find the source code from
> the 1986 version of CallText. They did, however, find the code for the
> upgraded 1996 version of the voice, on a backup tape in an office in
> Belgium. After a few months of work, Nuance engineers got the code up
> and running and sent a series of audio samples to Hawking’s team,
> adjusting the program to try to match the 1986 voice.
>
> It didn’t quite work. For one thing, the match was close but not
> perfect. Hawking flagged subtle differences others had trouble
> discerning. “It’s like recognizing your mother’s voice,” Price said.
> “When you hear them over the phone, they say two syllables and you know
> if that’s right or not.”
>
> The other issue was that Nuance owned the code, not Hawking. The famed
> physicist had always been intent on controlling the use of his own
> voice. If the team avoided using proprietary software, Hawking was
> likely to have more control.
>
> --------------------------------------------------------------------------------
>
> At this point, they switched tacks and returned to one of their original
> ideas: to emulate the CallText in software, similar to how PCs can
> emulate old Nintendo games that aren’t sold anymore.
>
> The CallText, of course, was a more intricate beast than a Nintendo,
> driven by two obsolete and complexly interacting chips, one made by
> Intel and the other by NEC. Building the emulator demanded heroic feats
> of programming, intuition and high-tech surgery. The chips had to be
> removed from a spare CallText board with tweezers and a screwdriver. An
> emulator for the Intel chip had to be written from scratch, by Benie. A
> separate emulator, for the NEC, was borrowed from an open-source
> Nintendo emulator called Higan.
>
> Then all these disparate pieces had to be glued together. It was a
> little like doing a jigsaw puzzle in a dark room. One chip was passing a
> mysterious packet to the other every 10 milliseconds. Why? What was in it?
>
> For a while, it was tough going. Some of the audio samples were so poor
> that no one dared play them for Hawking.
>
> The breakthrough came just before Christmas 2017, when the emulator
> finally started producing sounds that resembled the familiar voice they
> had been chasing. It had some minor glitches, but according to Price,
> the voice was an acoustical match to Hawking’s, the waveforms virtually
> identical. The only perceptible difference was a lack of analog buzz.
> “It’s like a clean and shiny scrubbed-up version of his voice,” Price says.
>
> When Benie heard it for the first time, coming out of a computer instead
> of Hawking’s voice box, he thought it sounded more American than
> Hawking’s voice. It was just an aural illusion. Benie realized that
> perhaps, whenever he saw Hawking speak, he had been mentally adding a
> hint of Britishness.
>
> Over the next weeks, in Cambridge and Palo Alto, the team members
> continued to debug the new voice, feeding it snippets of old Hawking
> speeches and sample texts full of random commas, listening to the results.
>
> On Jan. 17, the team felt ready to demonstrate the new voice for
> Hawking. Wood, Wozniak, and Benie went to Hawking’s home in Cambridge
> and played him samples on a Linux laptop. To the team’s relief and
> happiness, Hawking gave his blessing. It did sound like his voice.
>
> They still needed to port the voice to the PC, so temporarily, Wood
> loaded a version of the voice onto a miniature hardware board known as a
> Raspberry Pi. He thought Hawking might want to evaluate the voice in
> everyday life, and the Pi was the quickest way to get him up and running.
>
> On Jan. 26, Wood took the Pi along to Hawking’s house and asked if he’d
> like to try it out. Hawking raised his eyebrows, which meant “yes.”
>
> The team put the Pi in a tiny black box, attached it to Hawking’s chair
> with Velcro, and plugged it into the voice box. Then they disconnected
> the CallText. For the first time in 33 years, Hawking was able to speak
> without it.
>
> Wood watched eagerly for Hawking’s reaction.
>
> “I love it,” Hawking said.
>
> For the next few weeks in private conversations, Hawking continued to
> speak through the emulator and the Raspberry Pi, chatting happily with
> friends and colleagues. Wood said, “It was a pleasure to be able to give
> him something like that, that so many people have worked on for so many
> years.”
>
> All that remained, the final step in the project, was to get the PC
> version, still a bit buggy, working smoothly. But after a few more code
> revisions, it was finally bug-free.
>
> “We had pretty much completed all the technical hurdles,” Dorsey said.
> “Everybody felt, finally, this is it, it’s going to work, this is done.”
>
> And that is when Hawking got sick, in February.
>
> --------------------------------------------------------------------------------
>
> According to Wood, Hawking continued using the emulator until his final
> days. He was able to talk with his loved ones and caregivers with the
> new software on the Raspberry Pi. The last words he spoke while wired
> into his chair, whatever they were, he spoke with a version of his voice
> that lives only in code, potentially deathless bits and bytes.
>
> Everyone on the project understood that Hawking might not live long
> enough to get much use out of the emulator. He had been sick before, but
> always recovered. In 2014, when Wood first contacted Dorsey, Hawking was
> 72. They decided, though, that his CallText boards could keel over in
> six months, while Hawking might live to 80.
>
> Along with sadness at Hawking’s death, Dorsey can’t help feeling some
> disappointment. He and the team had raced for years to build a
> complicated thing that had worked beautifully, but now sat idle.
>
> At the same time, the project brought him back to his younger self, the
> guy who wanted to use engineering to perform good deeds and help people.
> All those years ago, working on the intonation algorithm in the
> CallText, he couldn’t imagine that it would end up helping define a
> genius of science to the world, and even to himself.
>
> Tech changes fast. Most machines end up as dust, and when we die, our
> voices die with us. Hawking’s voice is different. The original CallText
> boards have passed to his estate, to use as his family wishes. So has
> the new software, the CallText emulator, which can be ported to future
> platforms as they are invented.
>
> Hawking was, famously, an atheist, skeptical of the afterlife; “We have
> this one life to appreciate the grand design of this universe,” he once
> said, “and for that, I am extremely grateful.” But there is no longer
> any physical reason his voice can’t live forever.
>
> _______________________________________________
> Dectalk mailing list
> Dectalk at bluegrasspals.com
> http://bluegrasspals.com/mailman/listinfo/dectalk
> _______________________________________________
> Dectalk mailing list
> Dectalk at bluegrasspals.com
> http://bluegrasspals.com/mailman/listinfo/dectalk
>
>
>
> _______________________________________________
> Dectalk mailing list
> Dectalk at bluegrasspals.com
> http://bluegrasspals.com/mailman/listinfo/dectalk
>
>


-- 
“Be what you are. This is the first step towards becoming better than you are.”
– J. C. Hare & A. W. Hare