Tokenization: The Identity Crisis of the Word and AI History

20 min

Explore the history of AI from Joseph Weizenbaum's ELIZA to modern tokenization. Learn how computers process words through pattern matching and NLP history.

Tokenization: The Identity Crisis of the Word and AI History

How This Personalized Podcast Was Made

This podcast was created using BeFreed's AI, based on selected books, the creator's learning goals, and their preferred tone.

star
Input question

https://web.stanford.edu/~jurafsky/slp3/2.pdf

Podcast Style
Lenaplay

More like this

podcast cover
Direct source: blogs.nvidia.com
1 source
AI tokens and how they power the new digital economy
podcast cover
The Singularity Is NearerAI Snake OilSuperintelligenceThe Alignment Problem
28 sources
AI: Intelligence Without Understanding
book cover
Tokens
Rachel O'Dwyer
podcast cover
source 1The transformation from human surplus value to AI algorithmic surplus value: logic of the critique of capital in the era of AI | Humanities and Social Sciences Communicationssource 3How AI Is Commodifying Expertise and Dismantling the Architecture of Knowledge Work - Blog - Acalytica
6 sources
AI's Assault on Capitalism's Core
podcast cover
Digital Electronics Basics, Circuit, Uses, AdvantagesDigital Modulation Techniques - GeeksforGeeksUnderstanding Modern Digital Modulation Techniques | Electronic DesignDigital art - V&A
8 sources
The Logic of the Machine
podcast cover
Make Your Own Neural NetworkA Brief History of Artificial IntelligenceHow to Speak MachineHands-On Machine Learning with Scikit-Learn and TensorFlow
13 sources
Decoding AI: Beyond the Hype and Fear
book cover
To Be a Machine
Mark O’Connell
book cover
Co-Intelligence
Ethan Mollick

From Columbia University alumni built in San Francisco

BeFreed Brings Together A Global Community Of 200,000+ Curious Minds

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn
platform
star
star
star
star
star

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA
platform
comments
12
likes
117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw
platform
star
star
star
star
star

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum
platform
comments
12
likes
108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC
platform
comments
254
likes
17

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore
platform
star
star
star
star
star

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful
platform
comments
96
likes
4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP
platform
star
star
star
star
star

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon
platform
comments
201
thumbsUp
16

"It is great for me to learn something from the book without reading it."

@OojasSalunke
platform
star
star
star
star
star

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn
platform
comments
37
likes
483

"Makes me feel smarter every time before going to work"

@Cashflowbubu
platform
star
star
star
star
star

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn
platform
star
star
star
star
star

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA
platform
comments
12
likes
117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw
platform
star
star
star
star
star

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum
platform
comments
12
likes
108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC
platform
comments
254
likes
17

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore
platform
star
star
star
star
star

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful
platform
comments
96
likes
4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP
platform
star
star
star
star
star

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon
platform
comments
201
thumbsUp
16

"It is great for me to learn something from the book without reading it."

@OojasSalunke
platform
star
star
star
star
star

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn
platform
comments
37
likes
483

"Makes me feel smarter every time before going to work"

@Cashflowbubu
platform
star
star
star
star
star

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn
platform
star
star
star
star
star

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA
platform
comments
12
likes
117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw
platform
star
star
star
star
star

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum
platform
comments
12
likes
108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC
platform
comments
254
likes
17

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore
platform
star
star
star
star
star

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful
platform
comments
96
likes
4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP
platform
star
star
star
star
star

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon
platform
comments
201
thumbsUp
16

"It is great for me to learn something from the book without reading it."

@OojasSalunke
platform
star
star
star
star
star

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn
platform
comments
37
likes
483

"Makes me feel smarter every time before going to work"

@Cashflowbubu
platform
star
star
star
star
star
Start your learning journey, now