3CX MCU & MCU Manager Troubleshooting
Introduction
This guide includes a list of commands that tackle various issues you may encounter with the 3CX MCU and 3CX MCU Manager.
Restarting Services
Restarting the MCU
systemctl restart 3CXWMMcu |
Restarting the MCU Manager
systemctl restart 3CXWMMcuManager |
3CX MCU Appearing Offline
Check if the service is running by going to your Debian install and running the following command:
Systemctl status | grep mcu |
You should see the below 2 services running:
user@yourhost:~# systemctl status | grep mcu │ │ └─xxxx /opt/3cxwm/mcu/3CXWMMcu --pidfile=/var/run/3cxwmmcu.pid │ │ └─xxxx /opt/3cxwm/mcumanager/McuManager |
If the MCU services are not running, start them by following the MCU and MCU Manager restart procedure mentioned in the previous section.
- Firewall: Ensure your MCU and PBX can both reach each other. The MCU must be reachable on port 443, your PBX must be reachable on its configured port which is usually 443 or 5001 depending on what you selected during install-time.
- MCU PSK Settings: Go to your PBX System > Parameters > WEBMEETING_MCU_PSK, verify that the value within this parameter matches the parameter inside /opt/3cxwm/mcu/config.json called service_channel_secret. If it isn’t the same, copy the one from your PBX over to the config.json file in the MCU folder or vice versa, save and restart the MCU & the 3CX Gateway service on your PBX to ensure the changes have taken place. If the MCUs still appear offline, ensure you haven’t copied extra characters in the PSK when copy/pasting.
- Expired or invalid certificates: Ensure both your PBX & MCU have non-expired, valid certificates. In case you installed the 3CX MCU using option #1 “Use FQDN provided by 3CX and Let’s Encrypt SSL certificate” during install time, you’ll need to ensure port 80 of your MCU is open in order for certbot to be able to renew your certificate in time before its 90 day expiration. In case your certificate is already expired, you’ll need to go to your Debian install and type the following command to renew it:
sudo certbot renew |
In case certbot is not installed, you’ll need to install it by typing the following in your Debian 12:
apt-get install certbot |
For further information on how to use certbot please visit the Lets Encrypt or certbot websites.
- MCUs offline after temporary PBX failover: Ensure that the slave PBX’s 3CX Gateway Service is stopped when the master PBX returns online. If the service on the slave PBX remains active, MCUs will stay connected to it, preventing the master PBX from managing the MCUs. Enterprise customers should implement scripts to handle PBX services automatically, ensuring seamless MCU connections and avoiding conflicts.
Meeting Interruptions
- If PBX ←→ MCU connectivity is choppy and experiences constant interruptions, your meeting may be forcefully stopped by the MCU after a predetermined period of time of up to 1 hour.
- Restarting the 3CX PBX Gateway Service will interrupt all meetings on your PBX.
- Changing conferencing settings in your PBX Admin > System > Conferencing > Video page will also interrupt all meetings.
- MCU service crashes will also cause all meetings to be interrupted
- Lack of resources e.g memory or CPU will cause some users to be disconnected from meetings until enough resources can be recovered for normal MCU operations. Worst case scenario all users and meetings may be disconnected.
- Specific users e.g their IPs may be blacklisted by the MCU if they attempt operations that are considered malicious in nature by the MCU e.g like trying to access files on the MCU via HTTP/S. Blacklistings last for a few minutes for the first few attempts. Consistent attempts permanently blacklist users. Users will experience these blacklisting events as meeting disconnections and the inability to rejoin any meeting on said MCU. Whitelist your main usage IPs by SSH’ing to your MCU and editing the following file /etc/iptables/rules.v4.default be sure to add a rule to do so. Follow the instructions in Monitoring and Management
Inconsistent or Low Video Quality
- Low quality MCU VM or Provider
- Excessive CPU shaping on VM by provider or hypervisor
- No QoS assignment for MCU on network (e.g other network users stealing network time or bandwidth)
- User related issues e.g onboard or USB WIFI adapter requiring a restart, low quality network equipment, faulty network adapters, other software using all network resources, incorrectly strong WIFI reading when wifi strength is in fact low, low power CPU, insufficient system resources etc.
Automatic DNS entry deletion
- Unused OnBoard DNS entries (Ones provided for free by 3CX) will be automatically deleted. For this purpose, internal PBX APIs have been modified and now include & send the list of configured and active OnBoard MCUs on the PBX. This list is verified by WMR.3cx.net and unused DNS entries older than 30 days are deleted. This is necessary for the reason of keeping the my-3cx.net free DNS directory clean and free of abuse.
- If you’ve created the max (2) amount of MCUs allowable for your PBX FQDN and try to create a 3’rd one, you will not be allowed to do so. If you’ve uninstalled/reinstalled your PBX without restoring your previous (2) MCUs and you try to add more, you will still not be allowed. You will either have to restore your old backup & delete one or both MCUs in order to force your PBX to delete the FQDN from our DNS list OR wait for 30 or more days until the unused DNS records are automatically deleted on our side. Alternatively you’ll need to contact our support team and ask for a manual MCU FQDN deletion.
- If you install a PBX with a license that previously had two MCUs bound to it and then click the “Private Video Conferencing Server” button before restoring your backup containing your MCUs, this will send a reset command to our servers. This command will clear all 3CX-provided DNS records for any MCUs you previously had. Custom FQDNs are not affected as these are under your control.
Service Startup Failure or Service Crashes:
- MCU crashes randomly: The MCU requires 2 cores (2vCPU) minimum to run. It requires a dedicated VM to run on with the appropriate resources.
- MCU crashes randomly: Slow or problematic hard drives, insufficient memory or CPU, and excessive CPU shaping will also cause the MCU to crash. Excessive CPU shaping may appear as the MCU’s inability to use more than a certain amount of CPU.
- Check for missing folders: Make sure the Linux folder ’/var/lib/3cxpbx/WebMeeting/Docs‘. If they don’t, create them manually with the appropriate permissions.
- Check for missing, invalid or incorrect certificate format: Make sure your Linux ‘/opt/3cxwm/cert’ folder. You will have to manually add these and they must be fully valid, non expired, non self signed certificates authored by a valid certification authority. Also make sure the certificates are in the correct format which should be the .pem format.
- Check for missing certificate folder: Make sure your Linux ‘/opt/3cxwm/cert’ folder exists
- Set correct cert permissions: If the certificates cannot be read by the application which runs under a non root user, then the application will fail to start with a DTLS initialization error in the /var/lib/3cxpbx/WebMeeting/Logs/3CXWMMcu*.log files e.g
[DTLS_CTX] DTLS init failed, SSL_CTX_use_certificate_file error [ConnectionManager] DTLS initialization failed [ConnectionManager] terminated [main] Error while starting connection manager, the application will be terminated |
Ensure your cert files have the following permissions:
chown root:www-data /opt/3cxwm/cert/yourcertificate
chmod 644 /opt/3cxwm/cert/yourcertificate
Repeat this for all certs in the folder.
- Incorrect queue folder permissions: The permissions for the queue directory must be set to be owned by www-data with specific read/write permissions as follows:
chown -R www-data:www-data /opt/3cxwm/mcu/queue
chmod -R 700 /opt/3cxwm/mcu/queue
- Incorrect config.json permissions: The permissions for the config.json file must be set to be owned by www-data with specific read/write permissions as follows:
chown www-data:www-data /path/to/config.json
chmod 600 /path/to/config.json
- Check your config.json file:
- Where folders’ paths are specified inside your config.json file, ensure your folder paths are correct and that they exist.
- Ensure your IP address interfaces are correctly specified. Incorrect IP’s or non existing interfaces can cause the MCU not to start.
- Ensure your SSL and SRTP Ciphers are correctly specified.
- Ensure your ‘thread_pool’ values are equal or less to your vCPU cores (threads).
- Ensure that your config.json file is valid and not broken by pasting your config.json file contents inside a json format validator online such as: ‘https://jsonlint.com/’
If none of these steps fix your issue, create a new VM, create a new MCU entry on your PBX and install it on the new Debian 12 VM you just created.
FAQ / Frequently Asked Questions
- No recording “Start” button on Mobile Apps: Starting recordings from the mobile apps is not supported on the 3CX MCU.
- Dynamic IP is not detected on change: The 3CX MCU does not support dynamic IP detection. All IP’s must be static.
- What are the maximum MCUs allowed per PBX? Admins may add and install a maximum of 2 MCUs to their PBX system. This limitation is per PBX.
- How does the PBX select which MCU to connect to? (MCU Routing)In the case a PBX has 2 MCUs connected to it, the routing logic e.g selecting which MCU a meeting will take place is random. Meeting & MCU Monitoring Admins may monitor details of their on-premise MCUs and active meetings via the monitoring UI in their PBX > Settings > Conferencing > Video section.
See Also
- 3CX MCU Firewall Feature Overview
- 3CX MCU Network Configurations
- 3CX MCU On-Premise Meeting Recordings
- Quickstart - 3CX MCU Automatic Install
Last Updated
This document was last updated on 15 June 2024