There are three things to consider: settings, operation and connection.
Please check each in the following sequence:

1) Confirm the audio output destination setting
1. Open rekordbox Preferences.
2. Click [Audio].
3. Check that each item is selected as follows:

■ Windows
Audio: XDJ-RX3 ASIO
Mixer Mode: Internal
Master Output: XDJ-RX3 ASIO : MASTER
Headphones Output: XDJ-RX3 ASIO : PHONES
Mixer Mode: External
Output Deck1: XDJ-RX3 ASIO: CH1
Output Deck2: XDJ-RX3 ASIO: CH2

■ Mac
Audio: XDJ-RX3
Mixer Mode: Internal
Master Output : XDJ-RX3: MASTER
Headphones Output: XDJ-RX3: PHONES
Mixer Mode: External
Output Deck1: XDJ-RX3: CH1
Output Deck2: XDJ-RX3: CH2

*If you can’t select the correct choice because it isn’t displayed or an error has occurred etc., refer to [Troubleshooting -rekordbox) > "I can't set [AUDIO] in the rekordbox Preferences correctly."].

2) Check the volume level operation
1. Play a track.
2. Set the TRIM knob to the center (12 o’clock).
3. Slide the channel fader to the top.
4. Slide the crossfader to the center.
5. Turn the MASTER LEVEL knob to 2 o'clock.
If the MASTER LEVEL indicator lights up more than half way, there’s no problem in the settings and operation.
If you don’t hear any sound from the headphones at this point, check the following:
6. Turn the HEADPHONES MIXING knob to the center.
7. Turn the HEADPHONES LEVEL knob all the way to the left.
8. Press the MASTER CUE button.
9. Turn the HEADPHONES LEVEL knob slowly to the right.

3) Check the connection of speakers and headphones
For details, see [Connections] in the Quick Start Guide.
Even if you’ve already connected them, reconnect them to make sure they’re properly connected.
You can read the Quick Start Guide [here].

The installation of the driver software might be incomplete or the connection might be causing a problem.
If you’ve already installed the driver software, try again using the following procedure:

■Windows
1. Close rekordbox and disconnect the XDJ-RX3 from your computer.
2. Open the control panel (or [Start] > [Settings] > [Apps]) and uninstall [XDJ-RX3 Driver].
3. Close Windows.
4. Disconnect other USB devices, USB hubs, etc.
5. Start Windows.
6. Download the driver software from [here].
7. Install the driver software again.
　*An error will occur when the license agreement screen is displayed and you try to proceed without clicking "I agree".
8. Connect the XDJ-RX3 to another port without using a USB hub, etc.

If you still can't set [AUDIO], the problem might be caused by operating system settings, etc.
See case 3 and 6 in the guide [here].

■Mac
1. Close rekordbox and disconnect the XDJ-RX3 from your computer.
2. Download the driver software from [here].
3. Double-click [XDJ-RX3 Uninstaller] to uninstall it.
4. Close your Mac and disconnect other USB devices, USB hubs, etc.
5. Start your Mac.
6. Double-click [XDJ-RX3 AudioDriver.pkg] to reinstall it.
7. Connect the unit to another port without using a USB hub, etc.
8. Follow the instructions in the troubleshooting guide.
　macOS Big Sur: [here]
　macOS Catalina, Mojave, High Sierra: [here]
　
If you still experience the problem, it might be caused by operating system sound settings, etc.
Remove all other USB devices from your Mac, uninstall the virtual audio device etc, and restart the Mac.

The XDJ-RX3 might not be not properly connected, or rekordbox or your operating system might not be able to recognize the unit as a controller.

■ XDJ-RX3
Connect the unit using the following sequence:
1. Connect the XDJ-RX3 to your PC/Mac using the USB cable.
2. Press the SOURCE button, then press the SOFTWARE CONTROL button.
　　Set the [MIXER MODE] setting in the Utility settings if necessary.
3. Press the LOAD 1 and LOAD 2 buttons.
4. Start your DJ application.

If this doesn't improve the connection, try the following:

■ Operating system
Mac users, follow the procedures in the guide [here].

Windows users, follow the steps in Case 5 of the guide [here].

■ Others
You might solve the issue by doing the following:
- Disconnect any USB hubs and connect the unit directly to your PC/Mac.
- Disconnect any other USB devices.
- Quit all other applications.
- Connect the unit to a different USB port.

If the volume level is too high, the sound may be affected by distortion or clipping.

Play a track and follow these steps:

1. Check the channel level indicator
Adjust the level with the TRIM knob so the orange color lights up in the louder parts of the track.
2. Check the MASTER LEVEL indicator.
With the channel fader at the top, adjust the level with the MASTER LEVEL knob so the orange color lights up in the louder parts of the track.
3. Turn off MASTER TEMPO.

If distortion and sound clipping doesn’t improve, the output level might exceed the allowed input level of the connected speaker or amplifier. In this case, you need to lower the output level with the MASTER ATTENUATOR. For details, see [Changing the settings] and [MIXER category] in the [Utility settings] in the Operating Instructions.

If sound skipping, sound cutting, or noise occurs, see [Troubleshooting - rekordbox > "Sound skips, cuts, or is affected by noise."]

You can read the Operating Instructions [here].

Three main factors can cause this and there are three possible countermeasures:

■When buffer size is the cause
If the buffer size is too low, sound cutting or noise may occur.
You can adjust buffer size in rekordbox [Preferences] < [Audio] < [Buffer size].
(Windows users can adjust it in [Preferences] < [Audio] < [Setting Utility] < [ASIO tab].)
First, select a large value such as 1024. If the problem clears up, adjust to a value small enough that you don't experience a delay in operation (latency).
*The appropriate value depends on the system specification.

■When the connection or other equipment is the cause
An unstable connection with your PC/Mac might cause these problems.
To solve this, shut down your PC/Mac and disconnect all other USB devices and USB hubs. Then, connect the XDJ-RX3 directly to a different USB port.

■When your system (PC/Mac) is the cause
Doing the following might solve the issue:
- Quit all other applications.
- Disconnect from the internet and exit security software.
- Turn off wireless connections such as Wi-Fi or Bluetooth.
- Turn off power-saving settings.

If your computer's system doesn’t meet the requirements of rekordbox, it might cause the issue. See details on system requirements [rekordbox.com].

If you can't play audio files with rekordbox, check if the following applies:

• The file format is one that's not supported by rekordbox.
• Only certain applications can play the audio file, e.g. the audio file was downloaded from Apple Music, etc.
• The location of the audio file has changed.
• The audio file is damaged. For details, see [Management] and [Music files that can be loaded and played (file formats)] in the rekordbox Operating Instructions. You can read the rekordbox Operating Instructions [rekordbox.com].

When the XDJ-RX3 is used with rekordbox, the file formats that can be played differ depending on rekordbox modes.

For example, ALAC and high-resolution files can play only in Performance mode.
*In Performance mode, rekordbox loads and plays music. Therefore, you can play only files in formats that rekordbox supports.

For the file formats supported in Export mode, see [Supported file formats] in [Before you start] in the Operating Instructions.
You can read the Operating Instructions [here].

For the file formats supported in Performance mode, see [Music files that can be loaded and played (file formats)] in the rekordbox Operating Instructions.
You can read the rekordbox Operating Instructions [rekordbox.com].

Three main factors can cause this and there are three possible countermeasures:
■When the Windows built-in sound device setting is the cause
See case 4 in the troubleshooting guide [here].

■When the connection or other equipment is the cause
An unstable connection with your PC/Mac might cause this problem.
To solve this, shut down your PC/Mac and disconnect all other USB devices and USB hubs. Then, connect the XDJ-RX3 directly to a different USB port.

■When your system (PC/Mac) is the cause
Doing the following might solve the issue:
- Quit all other applications.
- Disconnect from the internet and exit security software.
- Turn off wireless connections such as Wi-Fi or Bluetooth.
- Turn off power-saving settings.

If your computer's system doesn’t meet the requirements of rekordbox, it might cause the issue. See details on system requirements [rekordbox.com].