Hide certain filters on specific categories
Catalog -> Categories -> Open some category settings -> Design -> Layout Update XML
Set something like
<body>
<referenceBlock name="catalog.leftnav">
<action method="setDisabledAttributeCodes">
<argument name="codes" xsi:type="array">
<item name="color" xsi:type="string">color</item>
<item name="size" xsi:type="string">size</item>
</argument>
</action>
</referenceBlock>
</body>