Você pode utilizar o apply e fazer um slicing da string
raw_data['nome_arquivo'] = raw_data['nome_arquivo'].apply(lambda x: x[:-4])
Pode também utilizar o replace
raw_data['nome_arquivo'] = raw_data['nome_arquivo'].str.replace('.txt','')
Entrada
ARQUIVOS
0 AAAAAAAAAAAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB.txt
1 AAAAAAAAAAAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB.txt
2 AAAAAAAAAAFFFBBBBBBBBBBBBB.txt
3 AAAAAAAAAAAFFFFFFFFFFFBBBBBBB.txt
4 AAAAAAAAAAAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB.txt
5 AAAAAAAAAAAFFFFFFFFFFBBBBBBBBBBBBB.txt
6 AAAAAAAAAAAFFFFBBBBBBBBBBBBB.txt
7 AAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB.txt
Saída
ARQUIVOS
0 AAAAAAAAAAAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB
1 AAAAAAAAAAAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB
2 AAAAAAAAAAFFFBBBBBBBBBBBBB
3 AAAAAAAAAAAFFFFFFFFFFFBBBBBBB
4 AAAAAAAAAAAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB
5 AAAAAAAAAAAFFFFFFFFFFBBBBBBBBBBBBB
6 AAAAAAAAAAAFFFFBBBBBBBBBBBBB
7 AAFFFFFFFFFFFFFFFFBBBBBBBBBBBBB