HP Wireless Printers cannot connect to WPA2-secured WiFi networks with Cisco/Meraki WAPs

[ttmcmurry]

In the last two months, I've had the displeasure of working with two very different HP printers and attempting to make them work on a WPA2-secured wireless network.  All attempts to authenticate fail with "invalid phassphrase". 

I'm not the first person to encounter this, it's a problem with many different HP wireless printers (I just happen to have physical access to the OfficeJet Pro 8610 & Deskjet 3511). 

My equipment is a Cisco ASA 5505 Firewall running ASA 9.1x & Cisco Aironet 1142 running IOS 15.3.x. 

What does work on the WPA2/AES SSID:  Apple MacBook Air running OSX 10.10.2, Three Windows-Based laptops running Windows 8.1 Update 1, an iPhone 5s, Three Windows Phone 8.1 devices, Roku 2, PlayStation 4, PlayStation 3, Sharp Aquos TV, Amazon Streaming Stick, and an Android Tablet (Jellybean).  Basically, everything. 

What does not work on the WPA2 network:  OfficeJet Pro 8610 & Deskjet 3511.

To test the theory there is a problem with HP's implementation of WPA2 with regard to Cisco Aironet IOS, I built out a second SSID that only works in WPA/TKIP mode.  This solution works.  Both HP printers will join the WPA/TKIP network.

So, I'm able to demonstrate there is a certain connectivity issue.  When i look at AAA Debug on the WAP's console, I can observe the HPs attempt to authenticate "Bind I/F" on the WPA2 SSID, however they do not achieve authentication and do not pass the AAA phase.  However, on the WPA SSID, they bind and authenticate successfully. 

To help illustrate this, here is my WAP running config.  It's about as simple as it can get.  There is no relevant MAC filtering or ACLs bound to any interface.  Noting that I have an ACL on remote access to the WAP (i.e. Locked down to SSH, disabling telnet).  The main point being that the ASA firewall is not a factor in this problem as the issue is at the WAP before WPA2 authentication can complete, therefore the printers never reach the network / when the printers connect to the WPA network, the operate fully & correctly. 

If anyone at HP can indicate why this particular config is somehow improper or broken, that would be fantastic.  There should be no reason why Cisco / Meraki WAP owners have to lower wireless encryption standards just for a printer, be forced into wired, create separate SSIDs with lower encryption specifically for a device. 

Building configuration...

Current configuration : 6064 bytes
!
! Last configuration change at 12:46:47 UTC Fri Aug 20 1993 by admin
version 15.3
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname 10-10-50-1
!
logging buffered 1024768
logging rate-limit console 9
!
aaa new-model
aaa authentication login default local
aaa authorization exec default local
aaa session-id common
no ip source-route
no ip cef
ip domain name freedom.local
!
dot11 syslog
dot11 vlan-name inside vlan 50
dot11 vlan-name inside-wpa-only vlan 70
!
dot11 ssid inside
   vlan 50
   band-select
   authentication open
   authentication key-management wpa version 2
   mbssid guest-mode
   wpa-psk ascii 7 xxxxxx
   information-element ssidl
!
dot11 ssid inside-wpa-only
   vlan 70
   band-select
   authentication open
   authentication key-management wpa
   mbssid guest-mode
   wpa-psk ascii 7 xxxxxx
   information-element ssidl
!
dot11 band-select parameters
   cycle-count 3
   cycle-threshold 200
   expire-supression 20
   expire-dual-band 60
   client-rssi 75
!
dot11 wpa handshake timeout 500
dot11 network-map
!
username ADMIN privilege 15 secret 5 xxxxxx
!
ip ssh version 2
bridge irb
!
interface Dot11Radio0
 no ip address
 no ip route-cache
 !
 encryption vlan 50 mode ciphers aes-ccm
 !
 encryption vlan 70 mode ciphers aes-ccm tkip
 !
 ssid inside
 !
 ssid inside-wpa-only
 !
 antenna gain 0
 mbssid
 speed  basic-1.0 basic-2.0 basic-5.5 basic-11.0 6.0 9.0 12.0 18.0 24.0 36.0 48.0 54.0 m0. m1. m2. m3. m4. m5. m6. m7. m8. m9. m10. m11. m12. m13. m14. m15.
 channel 2412
 station-role root
 l2-filter bridge-group-acl
!
interface Dot11Radio0.50
 encapsulation dot1Q 50 native
 no ip route-cache
 bridge-group 1
 bridge-group 1 subscriber-loop-control
 bridge-group 1 spanning-disabled
 bridge-group 1 block-unknown-source
 no bridge-group 1 source-learning
 no bridge-group 1 unicast-flooding
!
interface Dot11Radio0.70
 encapsulation dot1Q 70
 no ip route-cache
 bridge-group 70
 bridge-group 70 subscriber-loop-control
 bridge-group 70 input-address-list 700
 bridge-group 70 output-address-list 700
 bridge-group 70 spanning-disabled
 bridge-group 70 block-unknown-source
 no bridge-group 70 source-learning
 no bridge-group 70 unicast-flooding
!
interface GigabitEthernet0
 no ip address
 no ip route-cache
 duplex auto
 speed auto
!
interface GigabitEthernet0.50
 encapsulation dot1Q 50 native
 no ip route-cache
 bridge-group 1
 bridge-group 1 spanning-disabled
 no bridge-group 1 source-learning
!
interface GigabitEthernet0.70
 encapsulation dot1Q 70
 no ip route-cache
 bridge-group 70
 bridge-group 70 spanning-disabled
 no bridge-group 70 source-learning
!
interface BVI1
 mac-address xxxx.xxxx.xxxx
 ip address 10.10.50.1 255.255.255.0
 no ip route-cache
!
ip forward-protocol nd
ip http server
ip http authentication aaa
no ip http secure-server
ip http help-path http://www.cisco.com/warp/public/779/smbiz/prodconfig/help/eag
ip route 0.0.0.0 0.0.0.0 10.10.50.2
!
logging history size 100
!
access-list 111 permit tcp any any neq telnet
bridge 1 route ip
!
line con 0
 access-class 111 in
line vty 0 4
 access-class 111 in
 length 0
 transport input ssh
line vty 5 15
 access-class 111 in
 transport input ssh
!
end

[RenaudA]

I get the same behavior with a laserjet m451nw. I need to enable tkip to get the printer working, it doesn't support pure aes-ccm (every other device here supports pure aes-ccm, even cheap ones), although it's advertised as working.

The following snippet of config works, but I still think it should work without the tkip "hack".

dot11 ssid whatever
vlan 1
band-select
authentication open
authentication key-management wpa version 2


interface Dot11Radio0

encryption vlan 1 mode ciphers aes-ccm tkip 

[ttmcmurry]

That's in essence what I had to do and I didn't like it at all. On the firewall side, I restricted it to its own vlan with permitted ports to specific subnets only.

Its very insecure and not ideal for any environment. Imho bad security stewardship.

[RenaudA]

However, with the config I have shown before, when I do a "sh dot11 associations M.A.C", I get:

Key Mgmt type     : WPAv2 PSK          Encryption       : AES-CCMP
Current Rate      : m7t2               Capability       : WMM ShortHdr ShortSlot

 Which seems to mean it is connected correctly in  AES-CCMP. So perhaps it's connected in AES after all, but it cannot connect if TKIP is disabled.

[ttmcmurry]

I can say I understand what you're doing and why it's working with the configuration as you have it.  Essentially you're proposing to any potential wireless client the ability to use a TKIP or AES cipher with a WPA2 key.  While the device ultimately is using a WPA2 key, the problem I have with it is there is no enforcement for other devices to use AES.  Until TKIP is removed from your cipher list, a few things happen:

1)  it is not possible to get to 300mbit link speed on an 802.11n network, 130 will be your limit so long as TKIP is a proposed cipher.  that is, if the device follows 802.11 strictly (in the Cisco world, they follow the standard strictly)

2)  the integrity of the network is compromised due to TKIP being exposed as a valid cipher - there is nothing stopping a lower security connection down to a WPA key.

Having said that...

Cisco put out a new IOS out on 5/7 and it seems to have addressed the problem but for anothe reason altogether.  And I've learned more about the HP's WiFi security restrictions since the original post.  Fault is still on both sides wtih Cisco and HP.

• 15.3.3-JBB is the new Autonomous IOS
• Includes a fix for Cisco Bug CSCur08813 which affects the WPA handshake M3 key during handshake for Windows 8
• Note:  This fixed my Surface 2's inability to connect to AES/WPA2 networks
• HP doesn't implement 802.11w or Cisco MFP Client Protection in the printer's wireless stack.  It is not possible to connect to an SSID that protects (encrypts) management frames on that SSID; where either "ids mfp client required" or "11w-pmf client required" are set, it must either be set to "optional" or "disabled" altogether.

What I'm unclear on with the 1140 WAP is whether or not MFP is enabled by default with "optional" as its paramater if the configuration is not specified on an SSID.  For what it's worth, if you manually enable MFP but don't otherwise specify optional/required, "optional" is the default if you don't speciffy it.  I set the SSID to disable MFP altogether and the OfficeJet Joined.  So the SSID config looks like this now:

dot11 ssid printernet
   vlan 70
   band-select
   authentication open
   authentication key-management wpa version 2
   mbssid guest-mode
   wpa-psk ascii 7 xxxxx
   information-element ssidl
   no ids mfp client

dot11radio0

encryption vlan 70 mode ciphers aes-ccm

ssid printernet

(the subinterface is standard fare, dot1q tagged vlan 70, bridge group 70)

And the OfficeJet

Address           : 6cc2.17xx.xxxx     Name             : NONE
IP Address        : 10.10.70.5         IPv6 Address        : ::
Gateway Address   : 0.0.0.0
Netmask Address   : 0.0.0.0            Interface        : Dot11Radio 0
Bridge-group        : 70
reap_flags_1        : 0x0     ip_learn_type       : 0x0       transient_static_ip : 0x0
Device            : unknown            Software Version : NONE
CCX Version       : NONE               Client MFP       : Off

State             : Assoc              Parent           : self
SSID              : printernet
VLAN              : 70
Hops to Infra     : 1                  Association Id   : 1
Clients Associated: 0                  Repeaters associated: 0
Tunnel Address    : 0.0.0.0
Key Mgmt type     : WPAv2 PSK          Encryption       : AES-CCMP
Current Rate      : m7-2               Capability       : WMM ShortHdr ShortSlot
Supported Rates   : 1.0 2.0 5.5 11.0 6.0 9.0 12.0 18.0 24.0 36.0 48.0 54.0 m0-2 m1-2 m2-2 m3-2 m4-2 m5-2 m6-2 m7-2
Voice Rates       : disabled           Bandwidth        : 20 MHz
Signal Strength   : -67  dBm           Connected for    : 3335 seconds
Signal to Noise   : 79  dB            Activity Timeout : 26 seconds
Power-save        : Off                Last Activity    : 34 seconds ago
Apsd DE AC(s)     : NONE

Packets Input     : 1680               Packets Output   : 953
Bytes Input       : 989264             Bytes Output     : 120087
Duplicates Rcvd   : 0                  Data Retries     : 146
Decrypt Failed    : 0                  RTS Retries      : 0
MIC Failed        : 0                  MIC Missing      : 0
Packets Redirected: 0                  Redirect Filtered: 0
IP source guard failed : 0             PPPoE passthrough failed : 0
DAI failed : IP mismatch  : 0             src MAC mismatch : 0             target MAC mismatch : 0
Existing IP failed :  0              New IP failed :  0
11w Status       : Off
Session timeout   : 0 seconds
Reauthenticate in : never

The printer now connects at a PHY rate of 65-72mbit/sec as it's limited to a 1x1 radio config.  However the OfficeJet lacks both Cisco MFP or 802.11x management frame protection features.  I don't know what makes this particular Cisco IOS release unique over the last few years of IOS software.  Simultaneously, for a device that is aimed at businesses, the lack of the most up-to-date security features is unsatisfying.  I should never have to downgrade my network's security to accommodate one device - especially if it was made anytime after the inception of 802.11i (2004) and 802.11x (2009). 

[ttmcmurry]

I had a moment today to further look at my dot11 config.

If a dot11 ssid has "11w-pmf client" specified in either optional or required, the HP OfficeJet will fail to connect to an SSID.  Only after removing the statement as well as specifying "no mfp client optional" does the printer connect to a given SSID.

As I've indicated from the beginning it's the way that HP (VxWorks) implements security in their WiFi networking stack.  The presence of enhanced authentication options in standardized Wifi implementations will prohibit the device from joing a wireless network.

Lastly, two notes on encryption:

1)  If I use 802.11x the strongest cipher allowed is 3DES.  In a day and age where the standards is becoming commonplace to use AES-only, and even NextGen AES with ECC (AES & its variants are a FIPS 140-2 requirement), these printers are unsuitable for Federal use.  3DES suffers from a man-in-the-middle problem where its effective "bitness" is 112-bits.  It's "suitable" but compute-intensive - AES is a better performer.

2)  If I use SNMPv3, the strongest hash is MD5 and the strongest cipher is DES.  MD5 suffers from well known vulnerabilities and is very easy to break (seconds-to-minutes to break).  DES (56-bit) can be broken in a matter of hours.  This also is not FIPS compliant and is not suitable for Federal use.

This is unfortunate - as HP makes networking products which completely support FIPS as well as modern security compiant features a network admin would expect.

The saddest part of this is VxWorks itself supports AES and has for many years (since at least 2010).  Whether that's a licensing issue for HP to figure out is left to be seen, however there's no practical reason why in this day and age these features aren't present. 

[RenaudA]

@ttmcmurry

Thank you for your hint about 15.3.3-JBB, unfortunately that release is completely broken on my 3702e AP. It causes more issues than it solves. It seems that with this release, multiple clients are unable to connect or disconnect frequently. Furthermore, after some time (it seems it's dependent on the quantity of data), you get a huge latency hit (like 9000ms) and you need to reload the AP to solve the issue.

Also, the connectivity problem with windows 8 tablets/phones has shown on multiple AP brands, so I guess it's more a MS problem, just like the HP printer problem is more with HP.

[RenaudA]

It seems ap3g2-k9w7-tar.153-3.JBB1.tar solves the aes issue for me. It has been stable since I installed it.

[garygeo]

I had a devil of a time thinking the passphrase was right.  When going from the question on the scrolling window "WAP PASSPHRASE? I used the right button.  I think that made my first character in position 2 instead of position 1.  On the 7th try I got the 1st character ALL the way to the left of the window, and it worked.