Friday, July 30, 2010

How to clean up a firewall rulebase

1. Define a zone-based compliance policy and check it by running an audit report.

2. Identify and reduce insecure rules using the Best Practices report, the Security Risk Report, and the PCI-DSS report if it is relevant for your organization.

3. Optimize performance:

  • Remove bad traffic and clean up the network. Notify server administrators about servers hitting the firewall directly with outbound denied DNS/NTP/SMTP/HTTP(S) requests as well as dropped/rejected internal devices. The administrators should then reconfigure the servers not to send this type of unauthorized outbound traffic, thereby taking load off the firewall.
  • Filtering unwanted traffic can be spread among firewalls and routers to balance the performance and effectiveness of the security policy:
    • Identify the top inbound dropped requests that are candidates to move upstream to the router as ACL filters. This can be time consuming, but it is a good method for moving blocks upstream to the router and saving firewall CPU and memory.
    • If you have an internal choke router inside your firewall, also consider moving common outbound traffic blocks to your choke routers, freeing more processing on your firewall.
  • Remove unused rules and objects from the rule bases.
  • Reduce rule base complexity – rule overlapping should be minimized.
  • Create a rule to handle broadcast traffic (bootp, NBT, etc.) with no logging.
  • Place the heavily used rules near the top of the rule base. Note that some firewalls (such as Cisco Pix, ASA version 7.0 and above, FWSM 4.0 and certain Juniper Networks models) don’t depend on rule order for performance since they use optimized algorithms to match packets.
  • Avoid DNS objects requiring DNS lookup on all traffic.
  • Your firewall interfaces should match your switch and/or router interfaces. If your router is half duplex your firewall should be half duplex. If your switch is 100 Mbit your firewall interface should be hard-set to match your switch; both should most likely be hard-set to 100 Mbit full duplex. Your switch and firewall should both report the same speed and duplex mode. If your switch is gigabit, your switch and firewall should both be set to auto-negotiate both speed and duplex. If your gigabit interfaces do not match between your firewall and switch, you should try replacing the cables and patch panel ports. Gigabit interfaces that are not linking at 1000 Mbit full duplex are almost always a sign of other issues.
  • Separate firewalls from VPNs to offload VPN traffic and processing.
  • Offload UTM features from the firewall: AV, AntiSpam, IPS, URL scanning.
  • Upgrade to the latest software version. As a rule of thumb, newer versions contain performance enhancements but also add new capabilities, so a performance gain is not guaranteed.

Tuesday, July 20, 2010

The hierarchy of success

I think it looks like this:

  1. Attitude
  2. Approach
  3. Goals
  4. Strategy
  5. Tactics
  6. Execution

We spend all our time on execution. Use this word instead of that one. This web host. That color. This material or that frequency of mailing.

Big news: No one ever succeeded because of execution tactics learned from a Dummies book.

Tactics tell you what to execute. They're important, but dwarfed by strategy. Strategy determines which tactics might work.

But what's the point of a strategy if your goals aren't clear, or contradict?

Which leads the first two, the two we almost never hear about.

Approach determines how you look at the project (or your career). Do you read a lot of books? Ask a lot of questions? Use science and testing or go with your hunches? Are you imperious? A lifehacker? When was the last time you admitted an error and made a dramatic course correction? Most everyone has a style, and if you pick the wrong one, then all the strategy, tactics and execution in the world won't work nearly as well.

As far as I'm concerned, the most important of all, the top of the hierarchy is attitude. Why are you doing this at all? What's your bias in dealing with people and problems?

Some more questions:

  • How do you deal with failure?
  • When will you quit?
  • How do you treat competitors?
  • What personality are you looking for in the people you hire?
  • What's it like to work for you? Why? Is that a deliberate choice?
  • What sort of decisions do you make when no one is looking?

Sure, you can start at the bottom by focusing on execution and credentials. Reading a typical blog (or going to a typical school for 16 years), it seems like that's what you're supposed to do. What a waste.

Isn't it odd that these six questions are so important and yet we almost never talk or write about them?

If the top of the hierarchy is messed up, no amount of brilliant tactics or execution is going to help you at all.

Friday, July 9, 2010

Back to Basics – Troubleshooting Slow Applications (by Chris Greer)

Slow applications, who doesn’t have them from time to time? Even networks that are in tip-top shape can have issues in application performance. On many occasions when we start troubleshooting a slow application there is a push to immediately start looking in the SQL Traffic or SMB Calls or some other app traffic to determine the root cause of the issue. We find though that this approach takes us too deep, too fast. Assumptions are made about the network and the server itself that may throw us down the wrong path of troubleshooting, wasting time trying to blame the wrong component of the application.

When we start to troubleshoot a slow application, our first priority is to narrow down what bucket is causing the problem: the network, the client, or the server. Once we have identified where to place the blame, we can then get into the details to determine exactly what variable in that specific bucket is the root cause of the performance issue. Often, isolating the problem to one of these areas takes us back to the basics of the OSI model.

After we have successfully captured an example of the application running poorly (this is a VERY key part of the process – more articles on how to do this in the future…) we can use the rules of the transport layer to divide the OSI model in half, which helps us to place the blame on either the network or the server(s).

Osi
How can we do this at a packet level? We can look for two things: 1. TCP Retransmissions and 2. TCP Connection timers.


1. TCP Retransmissions

If these are observed in the trace file between client and server, then we can initially take the blame off the server for the cause of the performance problem. The reason we see TCP retransmissions is because the network is dropping packets somewhere along the path. This packet loss must first be tracked down and resolved then we can move forward with analyzing the server performance. With this one variable at the transport layer, we can cut the OSI model in half, leaving layers 5-7 alone, while focusing on the physical, data link, and network layers to see where traffic is being dropped.

How can we tell if there are retransmissions in the trace file? In Wireshark, use the Expert Info feature under the Analyze menu option. This window can be used to quickly see if there are TCP Retransmissions present in the trace file.


Wireshark



Note: Some client applications use TCP Keep-Alive packets to make sure that connections are not dropped between client and server. These may be interpreted by Wireshark as retransmissions. These are typically small packets that are repeated several times per minute. They are rarely a symptom of packet loss on the network.


2. TCP Connection Timers

After looking for retransmissions, we then analyze the connections between client and server to see if there is any delay at the transport layer when the handshake first happens. To do this, use the TCP Stream filter to isolate a TCP connection between client and server.


Tcp_handshake



Using the delta time column during the TCP handshake at the beginning of the connection, we can look for any delays in the connection setup time. If there are delays, these may be caused by packets being held up somewhere on the network, or by the server being slow in responding to the connection request. We may choose at this time to monitor the network between client and server, looking for excessively utilized links, or we may move our analyzer to the server to get a server-side view of the TCP connection setup. If the delay between SYN and SYN-ACK is still seen, then the server is holding up the show.

By starting with the transport layer when troubleshooting an issue, we can quickly cut the OSI model in half. If retransmissions are observed, we should focus on layers 1-3 to see where packets are being dropped, or if TCP connections are held up, we can focus on this layer on the server. Don’t get too deep in the application before ruling out the basics.
What do we do when we have ruled out the network? We will cover this in the workshop at Interop Las Vegas 2009 (shameless bait) or in future articles on LoveMyTool!



Wireshark TCP Protocol Sequence Number Customized (by Ray Tompkins)

In the Under the Hood technical article we’re moving onward in the discussion how to customize protocols within the Wireshark Analyzer. This simple but helpful tip is to turn on the TCP Analyze Sequence Numbers. This feature will convert the ACK and sequence number into a much easier value for us to interpret.

Go into the Preference menu and opening up the Protocols where you’ll find the known protocols that Wireshark decodes, see Figure 1:1. Move to the TCP and there’s the option to turn on “Analysis TCP sequence numbers”. In Figure 1:2 you see the results in packet with the sequence number changes from 1486019313 to 1.



Sequence Number

When a TCP session is established, indicated with the SYN, SYN ACK, ACK packets, the first TCP packet contains both the common port and the ephemeral port, “meaning short lived” and sequence number. There are also Flags, Window Size, checksum and options, and we’ll review those in the future. TCP uses the sequence and acknowledgment numbers to track the receipt of data bytes. It’s a random number for the purpose to reduce the predictability of the next TCP connection’s sequence number.

So in Figure 1:1 in packet 3 the initial sequence number is 1486019312 the corresponding ACK the response in packet 4 is Seq=1486019313. Wireshark has created an easier way reducing the initial sequence number to ZERO. It makes is easier to track as you look through the packets in the trace file. In example Figure 1:2 viewing the same ACK and sequence number we see how much easier it is, with the Seq=0 and the Ack=1.


Figure 1:1 Wireshark Edit>Preference>Protocols>TCP (Analysis TCP sequence numbers)


Wireshark TCP Sequence Number


Figure 1:2 Wireshark Edit>Preference>Protocols>TCP (Analysis TCP sequence numbers SELECTED)

Wireshark TCP Sequence Number  Selected

Top 10 Wireshark Filters (by Chris Greer)

6a00e008d9577088340133eca9c48a970b-800wi

The filtering capabilities of Wireshark are very comprehensive. You can filter on just about any field of any protocol, even down to the HEX values in a data stream. Sometimes though, the hardest part about setting a filter in Wireshark is remembering the syntax! So below are the top 10 display filters that I use in Wireshark. Please comment below and add any common ones that you use as well.

1. ip.addr == 10.0.0.1 [Sets a filter for any packet with 10.0.0.1, as either the source or dest]

2. ip.addr==10.0.0.1 && ip.addr==10.0.0.2 [sets a conversation filter between the two defined IP addresses]

3. http or dns [sets a filter to display all http and dns]

4. tcp.port==4000 [sets a filter for any TCP packet with 4000 as a source or dest port]

5. tcp.flags.reset==1 [displays all TCP resets]

6. http.request [displays all HTTP GET requests]

7. tcp contains traffic [displays all TCP packets that contain the word ‘traffic’. Excellent when searching on a specific string or user ID]

8. !(arp or icmp or dns) [masks out arp, icmp, dns, or whatever other protocols may be background noise. Allowing you to focus on the traffic of interest]

9. udp contains 33:27:58 [sets a filter for the HEX values of 0x33 0x27 0x58 at any offset]

10. tcp.analysis.retransmission [displays all retransmissions in the trace. Helps when tracking down slow application performance and packet loss]

Tuesday, July 6, 2010

McAfee AntiVirus deny telnet mailserver port 25

After invoking the Virus Scan Console, click on the properties of Access Protection. Under the "Ports to block:" uncheck the 'Rule' against 'Port 25' that says "Prevent mass mailing worms from sending mail".