convert: remove most of the n_mult usage in convert.py

[Green-Sky]

Little bit of clean up of n_mult. it is only used to calculate n_ff and the formula only works for llama and close derivatives. (eg not for falcon)

[KerfuffleV2]

Any reason not to remove the find_n_mult function? Nothing uses it after your changes.

[Green-Sky]

Any reason not to remove the find_n_mult function? Nothing uses it after your changes.

hm true. ... time to remove my hack then :)

[goerch]

Hm. A layman's view: I know of multiple_of as an input to compute the hidden dimensions, is this mapped to ffn_dim_multiplier now? I'd find it easier to follow if you'd be going to support different model classes to back reference the original models.

[Green-Sky]

Hm. A layman's view: I know of multiple_of as an input to compute the hidden dimensions, is this mapped to ffn_dim_multiplier now? I'd find it easier to follow if you'd be going to support different model classes to back reference the original models.

I am not sure where they come from, but yea, they AND the formulas describe a relationship in the architecture. But multiple_of can be a different value too, there are always multiple solutions. They also only seem to be specific to llama. falcon for example just uses 4 * hidden_size . Huggingface models also don't contain the information at all, they just have the intermediate_size param.