SW3 Crash/Reboot with mms streams to MPlayer based Clients

Miscellaneous updates and information for Smoothwall Express 3

Moderator: Forum Moderators

SW3 Crash/Reboot with mms streams to MPlayer based Clients

Postby wkitty42 » Sat Apr 05, 2008 8:06 pm

BoHiCa wrote:SW3 will throw a kernel panic and spontaneously reboot when viewing mms streams (Microsoft Multimedia Streams) in MPlayer derived clients running on Linux hosts behind SW3

Update 2 Status: This issue is *NOT FIXED*.

We just commented out the mms helper modprobes so they aren't loaded at all. Essentially, we applied the 'workaround' as described below as default since it is unacceptable to have a known vulnerability (DoS vulnerability at best, possible total pwnage at worst) in-place by default.

This *WILL AFFECT MMS STREAM VIEWING* basically, it breaks it for all clients behind the sw3. If you know, beyond any doubt that you will never have any MPlayer clients viewing mms streams behind your sw3, and you need to view mms streams on WINDOWS clients, you will need to load the two helper modules to be able to do so.


Summary:This is important because anything that can make the kernel sh!t the bed, is a security risk. This is intolerable on a peripheral firewall/router appliance. It represents a Denial of Service vulnerability at the very least, if not more with perhaps some cleverly crafted streams that potentially could allow for compromise and hjjacking of the appliance. We all love our buffer overruns!

*Note: It is important to realize that this issue *appears* to only be pertinent to users with Linux(tm) hosts that use the MPlayer (or derivative applications: mplayerplug-in, kmplayer, kmplayer-plugin) behind the firewall. If you only have winblowz clients behind the SW3, then you do not *appear* to be at risk from evidence gathered so far. Since the reality is that the Linux desktop, and the MPlayer application make up a tiny fraction of clients out there, this issue is not likely to affect very many users "in the world". Just be aware of the problem. But since most of us, here on the forums, are nerds, and do run some other flavors of Linux, we wanted to fully disclose the issue, and what we know about it so far.



More info:

1.) The kernel panic is EASILY reproducible through SW3 (see below for steps).
2.) The panic appears to be in the kernel NAT code, and perhaps not limited to just ip_nat_mms. I had at least one panic reporting that it occurred in ip_conntrack_mms, and several that did not report "user friendly" names where the panic occurred. That really sux, This means we really don't know where the issue is.
3.) It does not appear to be network driver related as the panic is reproducible with crap-nics, VM emmulated nics, and with Intel , 3Com and Tulip hardware.

Who this affects:
Potentially anyone that has Linux hosts using MPlayer derived clients to view any mms stream through the RED interface.

The crash is linked, somehow, to the mplayer plugin linux OS's. I could not reproduce the kernel crash with IE6/7 or Firefox 2.0.0.4, nor 2.0.0.11 on Winblowz XP SP2, Windows 2K3/2K server + all patches. (I do not run Vista in any form, so I could not test on that platform).
Code: Select all
Versions of MPlayer and plugins: (All versions are the latest I could find)
MPlayer: v1.0rc2
mplayerplug-in: 3.50
kmplayer: 0.10.0c
Firefox: 2.0.0.6, 2.0.0.11
Konqueror 3.5.7 (Using KDE 3.5.7 "release 72" openSUSE 10.3)


The crash has been reproduced with every desktop linux distro tested so far (OpenSUSE 10.1 (Live and full), FC6/7, PCLinuxOS, Ubuntu, Kbuntu, Mandriva (all modern versions I could find of the later 3). Basically, if the Linux OS can host MPlayer (or any of its derivatives), the crash will happen when attempting to view an mms stream.

The Workaround:
The problem *appears* to be limited to mms streams and the nat conntrack helpers required by netfilter to view them through a linux based firewall. Therefore the workaround is to simply remove those modules from the kernel.

Impact:
Removing both ip_nat_mms and ip_conntrack_mms appears to resolve the issue, however it GANKS all mms streams, they simply won't load (in any client), but the kernel doesn't crash.

HowTo: remove ip_nat_mms and ip_conntrack_mms
Option A: From the command line

1.) Log into the smoothwall as root, either at the console or via SSH.

2.) Once you are at the prompt, type these commands, exactly as written, hitting the enter-key at the end of each line.
Here's the commands:
Code: Select all
rmmod ip_nat_mms
rmmod ip_conntrack_mms

There is normally no output on the command line after each command.
That will remove the offending conntrack helpers while the smoothwall is up. No reboot required. You can verify the modules are removed by reviewing the output of
Code: Select all
lsmod


Removing ip_nat_mms and ip_conntrack_mms from the Start-Up scripts:
Perform the following to prevent the ip_nat_mms and ip_conntrack_mms modules from loading during a normal startup.

Note: Line numbers may be different if you have installed mods that twiddle with rc.network.

File: /etc/rc.d/rc.network.
Line: 47
Original: /sbin/modprobe ip_conntrack_mms
Change to: #/sbin/modprobe ip_conntrack_mms

and

Line: 48
Original: /sbin/modprobe ip_nat_mms
Change to: #/sbin/modprobe ip_nat_mms
(note the # symbol added to the front)

Save your changes. Next time the firewall is rebooted, the offending module will not load up. (But to confirm, you should check with lsmod at least once, as it may get auto-loaded by something else later on in the start up sequence if you have mods). I verified this on an unmodfied smoothwall 3, so if yours is modded, all bets are off, but the procedures should be similar).

For reference, this is the pertinent section from rc.network (Lines: 42 - 50)
Code: Select all
echo "Loading MASQ helper modules"
/sbin/modprobe ip_conntrack_irc
/sbin/modprobe ip_nat_irc
/sbin/modprobe ip_conntrack_ftp
/sbin/modprobe ip_nat_ftp
/sbin/modprobe ip_conntrack_mms
/sbin/modprobe ip_nat_mms
/sbin/modprobe ip_conntrack_pptp
/sbin/modprobe ip_nat_pptp


Modified to abort loading ip_nat_mms:
Code: Select all
echo "Loading MASQ helper modules"
/sbin/modprobe ip_conntrack_irc
/sbin/modprobe ip_nat_irc
/sbin/modprobe ip_conntrack_ftp
/sbin/modprobe ip_nat_ftp
#/sbin/modprobe ip_conntrack_mms
#/sbin/modprobe ip_nat_mms
/sbin/modprobe ip_conntrack_pptp
/sbin/modprobe ip_nat_pptp


Until we get a fix from the netfilter team, or find a later (or earlier), fixed, and compatible version of the ip_nat_mms module, this is probably the best way to handle this in the meantime.

If you must be able to view mms streams on Linux clients, then you will need to roll back to SW2, or use another firewall appliance/distro.

If indeed that is the top of the bug-tree, the firewall should *not* reboot when attempting to view an mms stream, but you may not be able to view the stream normally.

My suspicion is that the problem is slightly deeper down, in the NAT code itself. I seem to recall a discussion about a potential NAT bug that was fixed in later versions of the kernel (something in the 2.6.2x range if I recall correctly), and a newer version of iptables itself. That is where I will look next. Hopefully, I am wrong.


HowTo: Reproduce the Crash
Steps to Reproduce the Kernel Crash: (adapted from this post: viewtopic.php?p=199568#199568)

1.) Download and burn a bootable openSUSE 10.3 Live CD:
http://download.opensuse.org/distributi ... e-i386.iso

2.) Set up a bog-standard SW3. Services enabled are listed below. Only a RED (DHCP or static, don't know, and don't care about pppoe/a, serial modems, usb modems, nor ISDN :twisted: ) and GREEN nic enabled. It is safe to assume that any configuration is affected.
Code: Select all
Core services:
Logging server    running    1 hours, 05 minutes
DNS proxy server    running    1 hours, 05 minutes
Kernel logging server    running    1 hours, 05 minutes
CRON server    running    1 hours, 05 minutes
Web server    running    1 hours, 05 minutes
Services:
DHCP server    running    1 hours, 05 minutes
SIP server    stopped    
Quality of Service traffic shaping    stopped    
UPNP    stopped    
Clam Anti-virus server    stopped    
Web proxy    stopped    
Secure shell server    running    1 hours, 05 minutes
Intrusion Detection System    stopped    
IM proxy server    stopped    
Network time server    stopped    
POP3 proxy server    stopped    
VPN    stopped    


3.) Boot up the smoothie with at least one client behind it on the GREEN LAN. I had two live openSUSE's + 1 PCLinuxOS + 1 FC7 + 1XP-SP2 going. (I did not attempt to stream from multiple clients at the same time. All clients behind the SW3 were DHCP enabled and had IP/Gateway/DNS assigned by the SW3. The SW3 was directly connected to a hot RED link (e.g. no additional equipment between the RED and the modem (also tested with a T1 link. RED as configured with DHCP (cable modem test), and RED as static IP (T1 test) same outcome: crash).

4.) Once the smoothie is up and happy, boot up the Live distro CD on a client behind the SW3. I just let it boot up, it came up easily and without any fuss on all boxes I hosted it on. Quite nice really.

5.) Here's were it gets somewhat retarded:

Open up a super-user terminal window. Via the SuSE menu:
Applications | System | Terminal | Terminal Program - Super User Mode

You will need to add the packman repository to the YaST2 archives to install MPlayer, kmplayer, and the mplayerplug-in... In the super-user terminal window:
Code: Select all
linux:~ # zypper sa -n http://packman.unixheads.com/suse/10.3/ packman

You should then see this output:
Code: Select all
* Adding repository 'packman'
Repository 'packman' successfully added:
Enabled: Yes
Autorefresh: No
URL: http://packman.unixheads.com/suse/10.3/


You can leave the SU terminal open if you want to. For some f*cked up reason, when I attempted to use the command line YaST to install MPlayer, etc, the fekking keyboard stopped responding in the terminal session... this makes it impossible to accept the packman repo certificate... which pretty much defeats the purpose... so the "workaround" is to be a gimp and use the YaST2 UI :roll:

Via the SuSE menu:

Computer (3rd tab from the left) | Install Applications

Or you can double-click the "Install" icon on the desktop.

This launches the YaST2 UI. The initial load of YaST2 can take a bit, it seems to depend on your connection speed and the pc horsepower. Once it finishes it's bullshyte loading of repo info from both packman and the main SuSE repo, you can use the search box to find what you need to install.

6.) Search: kmplayer. It'll find two matching entries. Select the one for kmplayer alone (the debugging version might be neat, but not tonight). You will get a nasty-gram bitching about libspeex changing the "vendor" or some [censored]. Just click the option button to go ahead and change the vendor or whatever (it will make sense when you see it). Then click the "Accept" button. It will do the voodoo, download a crapload of dependencies, and install them all. I think the download is about 100 MiB. I installed the kmplayer first since it seemed to be the nasty actor, and doing this also installs MPlayer as well.

You can proceed to the bbc video page (listed below) now, to prove that kmplayer ganks the box, or you can install the mplayerplug-in first. It doesn't matter.

kmplayer = Konquerer support
mplayerplug-in = mozilla/firefox support

7.) Search: mplayerplug-in. Check the box, click the "Accept" button. This one is very small (~1.5 MiB) since the motherload of the dependencies was installed with the kmplayer above.

8.) TIME TO CRASH THE BOX:

a.) Open up a Konqueror session and click this link:

http://news.bbc.co.uk/2/hi/video_and_audio/default.stm

ANY of the video links (the ones with the "watch") button will bring the SW3 down.

The first time you click a "watch" button, you will get asked which player to view the video in. If MPlayer (and the required plugins) are installed, you can select the Winblows Media Player option. I did not check with the Real Player option, as I refuse to ever use Real Player for anything, but out of academic interest, perhaps it should be investigated also.

It can take anywhere from 30 seconds to 1-2 mins. The video will appear to start, but it never does.

The smoothwall will panic, and you will see the dump on-screen. The smoothwall automatically reboots after about 30 seconds, so you have to be pretty quick to catch it.

The kmplayer panics usually listed either the ip_nat_mms, or the ip_conntrack_mms modules as the source of the panic, and the Firefox mplayerplug-in intiated panics tended not to list any named modules (that I could decipher anyway). Perhaps there is a pattern, but with my testing, I did not discern one.

Related Links:
http://community.smoothwall.org/forum/v ... hp?t=26284
http://community.smoothwall.org/forum/v ... hp?t=26436
http://community.smoothwall.org/forum/v ... hp?t=26512
User avatar
wkitty42
solar system
 
Posts: 3731
Joined: Fri Mar 26, 2004 5:06 pm
Location: Central North Carolina, USA

Return to SWE3 Misc

Who is online

Users browsing this forum: CommonCrawl [Bot] and 0 guests