Hey guys,
I am a cyber security engineer and with my work I usually use claude with sub agents and skills to help me conduct my web and mobile application penetration testing.
Help me with some exploit development and research I do.
I want to try and do some of that locally;)
I have read a lot that fine tunning for your specific case will make the model much better and so on.
I need help so please bear with me and share with me your thoughts and prayers:)
I want to ask what models are recommended as base (I was thinking qwen 3.6 35b moe or qwen 3.6 9b dense (when it's released), I need very good agentic capabilities since almost all my usage will be over claude code)
I want to ask abou the data set and so on.
I don't have one yet:)
I recently got access to a private dataset on hugging face which has a little over 1 million rows.
The thing is, it's just text, not formatted to chatml or anything.
According to gemini i can use that text as post training data or something rather than fine tunning.
Would that work?
I also read that I can use a smaller model to create me chatml pairs or 3-turn agentic chats from the text to use it for fine tunning?
Recommendations please
And how many rows should the fine tunning be?
Also for training, should I use 4 bit or 16 bit:)
I will rent a RTX pro 6000 from vast.ai and use the q4km version of the model on my device.
I am really not sure what to do here as I am in no way an AI expert but I believe if I put enough effort to create an offensive security model.
I should get very good results with the needed privacy and a much lower cost on the longer run!
Your help and comments are much much appreciated!
[link] [comments]