Not atomic instructions

How should I work with non-atomic instructions? I`m on the PIC32 and if, for example, I want to set or clear an port then it is not an atomic instruction. How do I solve this? Should i place every time i want to change a port a taskEnterCritical / exit? There is a high possibility that another task also want to change something else on a port, sometimes.

Not atomic instructions

The PIC32 IO ports have separate set and clear registers that can be used safely without mutual exclusion because they do not require a read modify write type access.