how can i test my model using different dataset in machine learning

i m new in machine learning and i am create a one small project using CountVectorizer model. i am split my data to 80% -20%. 80% for training model and 20% for testing model. my model work properly run on 20% test data but can i used to test my model on different data set that is similar to training data set?

i am using joblib for dump and load my model.

from joblib import dump, load dump(pipe, filename)  loaded_model = load('filename') 

my question is how i directly test my model using different dataset?

Add Comment
1 Answer(s)

Yes, you can use the model to test similar datasets.

However, you must keep in mind the preprocessing step according to the model.

When you trained your model, it was trained on a particular dimension and the size of input would have been AxB matric. When you have a new test sentence or new dataset, you must first do the same preprocessing, otherwise, it will throw dimension mismatch errors.


suppose you have the following count vectorizer object

cv = CountVectorizer() 

then you must first fit it on your training dataset, for say

X = dataframe['text_column_name'] X = cv.fit_transform(X) # Fit the Data 

Once this is done, whenever you have a new sentence, say

test_sentence = "this is a test sentence" 

then you must use the cv object in the following manner

model_input = cv.transform([test_sentence]).toarray() 

and then you can make predictions:


This method must be followed even if you wish to test a new dataset which is in a data frame or some other file format.

Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.