Feature Request: Pre-reasoning safety evaluation using probability scores

Hello Momo I don’t have a background in tech either but I have been in the tech industry for 20 years building teams. Yes it’s possible and the thing is that a model normally have this probabilities at inference (when the model is creating your response) and you can increase or decrease the probability of specific tokens or concepts, or categories to be shown in the response however to do that you need to add some architecture from the ground up because mapping a already build model without “concepts” and only with tokens it’s super slow. If you have more questions and I know the answer happy to help!