Hello,

I am trying to collect traffic traffic from pcap file and store it in
a database but really confused how to organize it. Should I organize
it on connection basis/ flow basis or IP basis.

It might be an effort to write a customized traffic analysis tool like
wireshark with only required functionality. I would really appreciate
if someone can give me direction on write way of organizing the data
because right now I only see individual packets and no way of putting
them in some order.

Best,
Ali

Does this all not completely depend on what you actually want to do with
it? You might want to start there instead of the other way around.

Greets,
Jeroen

I want to be able to see information like how much traffic an ip send
over a period of time, what machines it talked to etc from this
perspective it should be IP based but I would really like to know how
other people do it.

Best,
Ali

Netflow   netflow collector.

Ken Matlock
Network Analyst
Systems and Technology Service Center
Sisters of Charity of Leavenworth Health System
12600 W. Colfax, Suite A-500
Lakewood, CO 80215

303-467-4671
matlockk*******

Run argus on a span port.

-Suresh

Random thought, anyone ever used Splunk for this kind of thing?

-mike

Sent from my iPhone

J�n�H¬jv¦k8§�Ú(�©Ýv'Z�¬²��¶§�©Ý�{k¡Û�yÖ�­ç

Various folks have, the problem of course comes down to processing
power, thus you'll need to throw a lot of hardware against it to be able
to process traffic in a decent network.

Check  http://www.raffy.ch/  and  http://pixlcloud.com/  etc for more
details about this.

Greets,
Jeroen

Run it with hadoop in EC2?

Sent from my iPhone

Truth is that most people probably don't do it, beyond temporary, ad-hoc
deployments, to solve a specific problem at a specific point in time.
Traffic capture and analysis doesn't scale too well into multi-Gb/s
service provider environments.

Netflow tools are an option if 'reasonably accurate' is good enough for
your needs.

jms

----------- myeaddress*******wrote: ----------
From: Maverick <myeaddress*******>

>> It might be an effort to write a customized traffic analysis tool like
>> wireshark with only required functionality. I would really appreciate

I want to be able to see information like how much traffic an ip send
over a period of time, what machines it talked to etc from this
perspective it should be IP based but I would really like to know how
other people do it.
-------------------------------------------------

Wouldn't Wireshark provide this for you?  In particular, the "Conversations"
tool under the "Statistics" drop down menu?  It adds data to the tool in
real time.  If you want a graphical output the I/O graphs also under the
"Statistics" menu can graph all, or slices of the data in the main
Wireshark output.

scott

Netflow / Sflow with one of the fallowing software packages

http://www.plixer.com/products/netflow-sflow/scrutinizer-net[..] http://www.solarwinds.com/NetFlow

http://www.arbornetworks.com/
Or the hand full of other open source options out there.

Carlos Alcantar
Race Communications / Race Team Member
101 Haskins Way, So. San Francisco, CA. 94080
Phone: +1 415 376 3314 / carlos*******

For high speed switched Ethernet environments, consider using sFlow.

You can treat sFlow as remote packet capture and use Wireshark/tcpdump
for troubleshooting network traffic:

http://blog.sflow.com/2011/11/wireshark.html

Or use sFlow reporting tools to find IP sources, protocols etc.:

http://sflow.org/products/collectors.php

Which tool to choose depends on your requirements.

PCAP is not well suited to what you describe. Most people use Sflow/Cflow/...
instead.

Owen

1 This guide should give you a good start.

http://techowto.files.wordpress.com/2008/09/ntop-guide.pdf

Regards

--
Mukom Akong Tamon
______________

"If we can't BREATH, we'll die. Yet, we don't LIVE in order to breath.
Ditto we SHOULDN'T WORK just to MAKE MONEY. Doing so puts us on a one
way street to IRRELEVANCE."

[In Search of Excellence & Perfection] -  http://perfexcellence.org
[Moments of TechXcellence] -  http://techexcellence.net
[ICT Business Integration] -
  http://ibiztech.wordpress.com
[About Me] -  http://about.me/perfexcellence

Thanks Mukom for the wonderful guide, this is really helpful. I have
few questions about ntop though.

How can I get access to the log files generated by ntop and do my own
parsing rather than looking for webbased results that are generated.
Are there any programs available that do parsing of ntops log files.
When I run ntop on pcap I don't get the throughput graphs as rrd
doesn't work on pcap is there any work around for that.

Thanks,
Ali

Hi Ali

It's been a while i looked under the hood of ntop. Remember that ntop
itself usually needs to be 'fed' traffic to analyse. I have never done
it myself but if I needed the raw data, I'd mirror a port and capture
it with tcpdump into a pcap file (watch disk space!!) the use whatever
analysis tool suits my needs to look at it.

Not to my knowledge no. I think there's a switch (-f) for reading data
from a pcap file as opposed to a live feed. I have never played with
that as well.

There are other (possible more feature laden) commercial flow
collectors and analysers out there). I also started following trisul
earlier on in the project, you might want to check it out.

--
Mukom Akong [Tamon]
______________

â=80=9CWe don't LIVE in order to BREATH. Similarly WORKING in order to mke
MONEY puts us on a one way street to irrelevance.â=80=9C

[In Search of Excellence & Perfection] -  http://perfexcellence.org
[Moments of TechXcellence] -  http://techexcellence.net
[ICT Business Integration] -
  http://ibiztech.wordpress.com
[About Me] -  http://about.me/perfexcellence