Requirement type classifier

Problem

Two labeling tasks on the same set of software requirement statements: is this requirement functional or non-functional, and was it written by a person or generated by an AI. The second question is the more interesting one, and the one that's aged into relevance rather than out of it.

Approach

Compared five text representations (TF-IDF, count vectors, Word2Vec, Doc2Vec, one-hot encoding) against five classifiers (logistic regression, decision tree, random forest, SVM, KNN), on a 400-row Turkish-language dataset, running the full grid rather than committing to one feature method up front. Preprocessing was standard NLP cleanup: lowercasing, stripping punctuation and digits, Turkish stopword removal, lemmatization.

Result

Sparse representations beat dense embeddings on both tasks, TF-IDF into logistic regression was the strongest combination: 0.80 precision / 0.64 recall on functional vs non-functional, 0.81 precision / 0.69 recall on human vs AI-generated. Word2Vec and Doc2Vec underperformed, most likely because 400 rows isn't enough data for embeddings to learn much useful structure.

What I'd do differently

Rebuild the human vs AI-generated detection task on a current dataset. Generated text has changed enormously since this was built, whatever separated "AI-generated" from "human-written" in this dataset almost certainly doesn't hold for text produced by current models. The functional vs non-functional task is more stable and would benefit less from a redo.