First: you open the def file of the character. you got that right?
Second: look for where it says:
pal1=
pal2=
pal3=
pal4=
pal5=
pal6=
Third: after any of the pal='s there is gonna be something that says (for example) EXAMPLE.act
change EXAMPLE for the name of the Act file (the palette)
Fourth: save the def file and open your mugen
Hope that helps