You have the answer here in the FAQ:
"The listen port for Remote Desktop Service itself also should be open. It uses TCP 3389.
The vMatrix Network Service uses both TCP and UDP ports. The broadcast port is fixed using UDP 13389, and the listen port takes 5 ports from 13389 to 13393 and 1 extra for each client user. The following example is given on 10 users (always reserve more ports if possible).
3389- 3389 192.168.x.x TCP
13389- 13389 192.168.x.x UDP
13389-13403 192.168.x.x TCP
These ports are default settings on vMatrix installation but able to be customized at vMatrix -> Configuration ->IP/TCP ports."
I think you are missing the UDP ports, and the extra 5 ports + one for each connection