Natural Language Processing With Python and NLTK p.1 Tokenizing words and Sentences

Share it with your friends Like

Thanks! Share it with your friends!


Natural Language Processing is the task we give computers to read and understand (process) written text (natural language). By far, the most popular toolkit or API to do natural language processing is the Natural Language Toolkit for the Python programming language.

The NLTK module comes packed full of everything from trained algorithms to identify parts of speech to unsupervised machine learning algorithms to help you train your own machine to understand a specific bit of text.

NLTK also comes with a large corpora of data sets containing things like chat logs, movie reviews, journals, and much more!

Bottom line, if you're going to be doing natural language processing, you should definitely look into NLTK!

Playlist link:

sample code:


Quintin SA says:

Hi sentdex how can i use nltk to create an application that marks essays?

Ravi Samal says:

watching in 2019

Alex Sipov says:

the way he greets is a copycat of The New Boston

shalini sahni mankatalia says:

You are da best….. 🙂

suwarna choudhary says:

Hi, on running above code I am getting error- TypeError: expected string or buffer
. I could not find any solution even on stackoverflow. Can I please get some help?

Shamoon Ahmad says:

Sir kindly make tutorials on these contents which are required for future with projects. Thanks??
Natural language processing using NLTK and Python

Module Contents:
• Basic text analysis with NLTK
• Text pre-processing
• Stopword removal
• Stemming and lemmatization
• Parts of speech tagging
• Chunking
• Named entity recognition
• Wordnet with NLTK
• Text classification
• Converting word to features
• Classifying text documents using NLTK
• Integration with scikit-learn classifiers
• Gensim
• Word2vec
• Investigating data biasness using NLTK
• Twitter sentiment analysis using NLTK

Nandish Ajani says: is not working.. It says module 'nltk' has no attribute 'download'

Shruthi gowda says:

Thanks for sharing these great videos, I am a software developer from Germany, currently I am on research on the topic "Natural language processing for SQL" for example: For a given Relation Student(matriculationNumber, name, course) the query "How much students study Web Engineering“ should translate to "select count(*) from Student where course=Web Engineering“. whats your input or guide on this? I hope I receive the response. Thanks again 🙂

Mouloud Aït-Kaci says:

"Ptdrrrrr" is like "lol" in french. It's an acronym for "Pété de rire" which literally translates to "Broken from laughing". This is from a lot of french people commenting on funny content.

Dinesh Kumar says:

hello i am trying to install nltk but during the download i am getting error saying that "out of date".
can you provide me the solution…..

krzysiek krzynowek says:

You are awesome 🙂

Sandipan Ghosh says:

I am not judging you, why do you use Windows? Why not Linux.

Sanjeeth Mv says:

My example_text is "Hello Mr. Bond, How are you doing today?"
This gets separated as ['Hello Mr.', 'Bond, how are you doing today?']
All other names seem to work fine.. why not in this case?

Jim Wright says:

Excellent tutorial. I'm curious though. You said that one of your tutorials needed updating badly. Did you ever update it badly? ; – )

Gels_YT says:

do you think idle is the best python ide? notice mee, thanks for the tutorials

Alina Aldea says:

can I use this for tweets in other language, and not english?

Taj kadar says:

Can u help in getting a clear picture to work on mixed data(continous, discrete and descriptive) how to go with Random Forest on top of NLP/ SVM with NLP. Like that plz.

Greb Jawson says:

holy crap, you have tutorials for everything

KoltTv says:

Why do you teach in Windows?

TommyTwobats says:

Can I use this as a tool to measure how 'natural' or 'typical' my English language learners' sentences are?

Subash Mahat says:

I wanted to learn NLP with python so that i could use it for sentiment analysis at my work. I was looking for a good tutor, now I found JESUS

Paul Newton says:

Any work on using this on programming languages or simpler dsl template systems?

shristi jaiswal says:

Give relevant examples than making it funny

Zahidul Islam says:

Can you please help me how to how to made language model and acoustic model dataset for speech recognition

Write a comment