November
Business continuity planning and
prepardness.
http://www.fraudconference.com
November 13, 2006
Conference on disaster recover, business
continuance planning.
December
Information technology and the law.
http://www.misti.com/
Dec. 9-10, 2006
This seminar will explore the legal issues surrounding
Internet use, the electronic workplace, information security, privacy
protection and incident response and much more.. For more information, please visit
the MIS website - link provided above.
Op911.Net
Staff
Concept and idea
Rod Deluhery
Executive
producer
Betty Jimenez
Editors
Susan Fitzgerald
Bob Hogue
Gary Helbrink “the radio man”
Printing and design by
West Koast Graphics
Publication and distribution
services by
Hitech LLC.
|
|
Buying knowledge?
By Rod Deluhery
A slight problem
The dispatch
supervisor looked at me with a tilted head and a sly smirk of disbelief. “What can we do to prevent this, what about .
. .” I interrupted her. I was trying to defuse a hot situation with
a slightly unhappy camper. The less than
satisfied public safety officer had just experienced a near loss of
functionality of a major 911 dispatch center, and I had some explaining to
do. In retrospect it was a learning
experience.
It all started with a slight problem, a glitch in the
system.
The dispatch center called our computer office, saying “the
network is slow”. For a large city
police dispatch center, this is not a good thing. It wasn’t a busy day, but the dispatch people
where still a bit annoyed at the problem.
Jim, the dispatch super gave me the weird look.
“You know, this is the same thing that was happening last
time, and it was the network. It wasn’t
a CAD problem”, Jim said.
“True, but I don’t see anything wrong. . .yet”. I said, a little worried but no panic yet.
I continued to look at the problem, thinking it was probably
the server or database. Finally I
started doing some more network tests.
“I think I found part of the problem. . .we are losing pings
on the network. It’s network related, at
least part of it, anyway.” I said.
Suddenly the problem was mine and my blood pressure went up.
I scrambled to find the problem. Using Ethereal, I captured some data on the
CAD network vlan, nothing there. Then I
captured data on the other ports of the switch.
Whoa, massive amounts of data flooding the network. Aha, this must be what is slowing it
down! But what was it, all this UDP
traffic? Soon the deluge of data
stopped, but dispatch was not quite satisfied.
“What can we do to prevent this next time?” the dispatch operator asked me.
“Well, we can discuss it, after I find out what it was. But there are some things we can do. One is to buy a dedicated network switch for
CAD users, and separate the CAD network.“
“How can we better isolate the problem next time? Jim asks, trying to find a resolution.
“Well, I can write up some things on how to better
troubleshoot these issues. Well I
suppose I could document some things to test next time you see this.” I say,
feeling a little better that I actually know something about the problem
now. What happened that made this
incident different than any other computer problem? The issue was it brought a metropolitan 911
dispatch center to a point where they could almost not dispatch calls. A very bad thing!
PREPARE FOR EMERGENCIES, AT HOME OR WORK WITH
LIFEGEAR PRODUCTS.
http://www.lifegearcompany.com/
Post mortem
People who understand technology realize there are many
parts that make up our complex machines and systems we use. Look at a helicopter, for example and compare
it to a large IT system. Both have many
interconnected, single parts that, if they fail, the system can be crippled,
unable to perform the mission at hand.
System engineers, technicians, and the operators that use the equipment
all have various levels of knowledge on how IT systems work. The question is, what level, or what training
can give users and operators enough information to help diagnose problems?
According to Albert Einstein:
“The world we’ve made as a result of the level of thinking
we have done thus far creates problems that we cannot solve at the same level
at which we created them.”
Basically this means we have created some monstrous machines
needing monstrous skills and training. As
a general rule, ignoring some theoretical discussion, technical training can be
defined as:
Risk = complexity
divided by training. As a formula, R =C/T, meaning the more training, the
less risk. And the more complexity, the
more exposed risk. Some engineers will
argue that they can build in complexity and redundancy that minimizes
risk. I’d argue that. The general tendency on many systems is to
build IT that only has a “red warning light” type signal that works similar to
a car warning engine light. Instead of
having five different gauges, the output is summed to one red light. From operational point of view, this makes it
easier for operators or technicians to be alerted that something is wrong,
meaning they have less complexity, less objects to keep track of. There is a downside though. This makes it difficult to identify what
exactly is causing the problem. As a
manager of technology, you need to keep aware of dumbing-down your staff into
just “red light watchers”.
The trend to simplify complex technology is going to
continue. In my humble opinion, such
simplification does not mean you can skimp on training. Operational expenditures (opex) will climb if
your staff does not understand how to troubleshoot complex systems. Saving money on theory of operation, and
diagnosis will end up costing you in downtime.
For your information, and for those who want the gory
details, the problem was a software problem in a network router. The exact situation in that a multicast
packet was being sent to all ports on a router and switch, instead of a single
switch. The resulting data traffic
slowed all the networks down. It wasn’t
apparent where the data was coming from until this incident occurred once more
and the event causing the multicast was identified. It was the computer staff imaging computers
with the Symantec Ghost program, which uses multicast.
|
Lessons learned.
 The take away
lesson from this:
|
Make sure your staff can diagnose those “red lights” quickly
and effectively. Examine the downtime
costs versus training costs to minimize service outages.
The invisible and silent bullet.
Here is a question.
What is invisible to the human eye, silent, can penetrate a building,
cause computer systems to crash, and blow fuses? No, it’s not a virus. The answer is radio waves! We all know that radio waves, or RF (radio
frequency) emissions can be dangerous. It’s common knowledge that the 800mhz signals
from our cell phones penetrate our brains and can cause cancer. But we all live with that risk, and play the
game of life taking the risk along with all the other chance exposures, like
crossing the street and hoping not to be run over by a truck. What is interesting to me is how we all have
come to live with these invisible waves, surges and swells of electrons that
pass through us everyday. What to worry,
they are part of modern civilization and it’s a cost of doing business,
correct?
One problem with the ignorance of exactly WHAT radio signals
surround us, is that we don’t have a baseline, and we ignore what is
normal. Similar to the adage of if you
don’t measure something, you can’t improve it.
Except here, the improvement we are talking about is to understand radio,
so that you can prevent interference which degrades performance. Or to understand radio so you can identify
who sent the radio signal that triggered the bomb. So what should we know about RF?
In my opinion, large organizations that rely on multiple
radios systems must have some radio engineer on staff. He may not need to be an EE with a degree in
RF design, but at minimum a radio tech or staff person, who can use a spectrum
analyzer and is familiar with the radio signals your organization uses.
Many radio systems are using the best effort, unlicensed
mode of operations. What does that
mean? It equals two words: GOOD LUCK.
That means if the system gets interference, don’t call the manufactures. Whether it is a vendor who makes an RFID
inventory scanner or a 2.4 ghz wireless phone, you have to deal with the
effects and try to mitigate the problems by either moving equipment, relocating
operations, or buying new equipment that doesn’t interfere with each other! Trying to troubleshoot communications
problems on the radio side, or physical layer can be difficult. If the problem is found to be interference,
the answer is mitigate the interference.
Radio licensing.
Who handles your radio licenses? Some emergency radio equipment is FCC
licensed and that license is held by your organization or another company. Who is keeping track of your licenses? If the license is held by a private company,
it would probably be a good idea to review your licenses once a year. The company could go out of business that
holds your licenses, or be aquired by another company that is going to raise
your rates. Also the public license you
now have could be up for sale by the FCC.
There is a licensing battle that has been going on for some
time between public agencies and the private companies. There are billions in stake and the private
companies want as much spectrum as they can get. You may need to lobby the FCC to keep your
licenses. Do you have someone that could
do that? These are things to think about
with radio communications.
|
Lessons learned.
The take away
lesson from this:
|
Ensure that there is someone who knows at least a baseline
of how your radio systems work, from a radio frequency standpoint. That means they need to know how far their
signals should travel, how often they get interference, and so on. For large organizations, an RF expert who
can diagnose interference, sources of radio emissions, and perform baseline
checks of radio equipment is mandatory.
Ability to use a spectrum analyzer (and have one on hand) is important.
Designate someone to keep track of you radio licenses and
any issues effecting your ability to communicate.
What
is in this box could save your life. http://www.life-box.org
Emergency Preparedness In The 21st Century
By Joseph J. Mack
Facility security planning, building security systems,
emergency preparedness coordinators, safety directors, and emergency plans are
accessible to almost everyone. Companies have provided fire extinguishers,
emergency exits, hose bibs and fire alarms.
Some facilities have even installed Co2 detectors in the
parking garages, and the appropriate gas sensors in the factories or
laboratories that we work in every day.
We have brave and well trained first
responders.
Recent events, both man-made and natural have shown
us that virtually every factory, plant, mine, apartment building, office
buildings and both sports and concert venues are at risk for a catastrophic
disaster.
The question now becomes what do your employees, guests or
visitors do until the first responders find them? How will they find them? Will
they search aimlessly thru crumpled structures for survivors, or will they see
flashes of light, or perhaps hear an alarm signal to direct them to those that
are trapped?
History has shown us it could be hours, it could even be
days before the rescue teams arrive to begin the extractions. How will those
who are trapped survive that time?
The essence of survival is the
"will to live." Unfortunately, the will to live is not enough. Good
survival equipment, along with training, are necessary for that “will to live”
to flourish.
Survival depends on our ability to deal successfully with the common
physiological crises of hypothermia, hyperthermia, dehydration, starvation,
fatigue and other environmentally induced physiological problems, and to do so
with limited, often nearly nonexistent, resources. The more resources you have
at your disposal, the better your chances are. It's that simple. More is
better.
Without training and equipment, you are utterly at the mercy of your
situation. Without the equipment, you had better be very well trained and
extremely lucky, because survival can be extraordinarily more difficult. It is
a lot more practical to be well equipped. After the incident it’s too late to
think about “what we should have done”. All of the investigative commissions in
the world won’t bring back lost loved ones, yet they will point the finger at
what should have been done so “it won’t happen the next time”.
Right now, today, all of those potential commission
members and critics have to look in the mirror and say “Now is the time!”
Be who you
think you are!
It’s now very apparent that the facility security planning,
building security systems, and emergency plans are woefully inadequate.
Emergency preparedness coordinators and safety directors need to be made aware
of the new tools available to them and be trained in the use of these tools.
And while fire extinguishers, emergency exits, fire alarms, Co2 detectors and
other gas sensors are necessary, they are not enough. The face of facility
security and personal safety has changed, and that change mandates that
all facility and safety professionals provide survival equipment to their
employees.
Don’t send another miner into the ground without being
prepared. Don’t ask another young mother to man her cubicle in a high rise
office building with no hope for survival in the event of a catastrophic
disaster. Don’t ask another child to attend school with no hope for rescue in
the event of multiple simultaneous events.
There are several options in the market place for you to
obtain these products. Keep in mind however that no matter which option you
choose, you will have to live with the consequences of that decision. Don’t cut
corners. Don’t be penny wise and pound foolish. Lives are at stake. It’s easy
to spend an extra couple of thousand for leather upholstery in your new car, or
an upgraded stereo. It’s easy to justify new flat screen monitors, interior
plants or a new copier for the office.
It should be a no-brainer to justify the very best in
facility survival equipment for your employees, friends, students and
co-workers.
How could you face their survivors if
you didn’t make that effort?
I’m asking you all now to make that effort. And while I’m
confident that the Life-Box is your absolute best alternative, anything is
better then nothing.
If you have any questions, feel free to contact the author
directly at jmack@life-box.org, and
please visit our web site, http://www.life-box.org
for additional information.
You owe yourself the good night’s sleep that comes from
knowing that you did all that you could.
Survey of police and sheriff radio.
A survey was done of 152 small and
rural police and sheriff agencies in all 50 states. The survey asked the organizations what
technology they had and what they didn’t have.
Respondents were asked to
indicate the extent to which different types of voice communications technology
are available to their agencies. Responses were as follows:
All Some Personnel
Mobile radios 6.5% 3.5%
Mobile phones 54.1% 38.8%
WiFi 7.5%
11.3%
Satellite phones ----
9.6%
Voice over IP 2.4%
6.1%
I come from a large police agency, where every officer has
a mobile radio (walkie-talkie), but looking at these numbers for small county
agencies shows that many DO NOT have mobile radios. It is an interesting case for more services
over commercial mobile phones, where a convergence of technology is ready. Convergence of web services, trunked radio,
and other rich media services can be delivered over existing commercial
cellular phones. Modern cellular phones
can provide map services, web services, connections to other voice systems, and
any other internal network system.
A common device that provides data and voice is the
Blackberry handheld. Blackberry phones
are an interesting example of convergence.
With a blackberry enabled phone, all data and web communications start
INSIDE the corporate network. That means
internal resources and websites are AUTOMATICALLY available to blackberry web
users, via the intranet blackberry server.
The blackberry server relays all data from the provider radio
network into the internal network. A
very valuable asset!
At the city I work at, blackberry’s are used to browse the
inside phone web directory page, ping devices inside the network, and many
other unique functions. At this shop they also use a computer aided dispatch
program called Versaterm. Computer aided
dispatch is usually called “CAD”, and the vendor is now gearing up to have a
small version of the program that will work as a Java applet on
Blackberrys! Having some CAD
functionality on a blackberry device is very handy for the officer on the street
who walks or bicycles.
|
The take away lesson from this:
|
Information is being delivered into many new devices and
that give the public safety person more options for mobility. When deciding the purchase of pagers, cell
phone technology or other devices, keep in mind what future functionality
people will be requesting.
Pandemic planning, and working from home.
Disaster recovery planning is
being challenged by the possibility of avian bird flu or other outbreak that
would leave workers stuck at home for weeks at a time. The worst case scenario is that avian bird
flu mutates into a fast spreading communicable disease. The challenge for public safety, county and
city services is this. How do you still
provide services when your staff can not leave home or is simply afraid to
enter into the normal workplace?
City jobs that require physical
labor will be severely impacted as workers call in sick. But the computer operations that many people
do all day long can be done from home. Remote
computing can assist in some situations.
The challenge is how to make it work securely and easily from users
homes. Remote computing can help when
they are either afraid to come to work or have symptoms of the disease
themselves. Remote computing, or
telecommuting, has been around for years and is used by many corporations. How can such technology be used?
For a brief example of how remote
computing would help an organization, we will start with a critical city
service, paying the bills! Verifying pay
records, verifying overtime and assimilating all the businesses and services
who need to get paid is a huge job. The
final product after the batch runs is to create the paychecks for distribution.
The city users who do this at their
computer would need to be able to do this from their home computer in a way
that is secure and gives the users access to the same resources they have in
the office workplace. What are the
challenges to doing this? The first is
to find out what are the most important tasks that keep the city running, like
check writing. Parcel out the tasks into
high and low priority, in order of how often and the amount of people it takes
to do the task.
One the tasks are broken into
priority and number of users, those applications that people use can be put on
remote access servers in such a way that most home based users could run the
application from their home pc. That
means minimal amount of software installed on the users home computer, and a
secure connection to the city resources either via internet, dial-up or
radio.
This would allow the users to run
these applications from the computers at home without endangering themselves or
others.
The technology exists to solve the
problems discussed here, the question is how much money are governments willing
to spend? Are they ready to spend money
on building systems that are only used for emergencies? There has been more than one local government
that recently stopped paying for offsite disaster recovery services, even in the
light of Hurricane Katrina. Setting
aside funds for disaster recovery systems is like setting aside money for
anything that is not critical. The money
is competing for many other hands and people are fighting for those funds.
Let’s go over some of the steps
needed for pandemic planning, and disaster recovery.
- Ensure
you incorporating lessons learned!
Every past disaster has lessons that should be used in revised
plans
- Incorporating
human impacts into the recovery and planning process
- Prepare
for the human nature aspect, to prepare staff for the psychological
effects
- Educate
and communicate with employees to increase their awareness
- Leveraging
executive and management buy-ins
- Testing,
and enhancing work from home procedures and options
- Merging
private and public sector best practices to enhance emergency preparedness
- Enhancing
communication techniques during disasters and measure the effects
- Building
business unit, and departmental preparedness to cross train employees
Key to building a good disaster
recovery system is to have a plan, take small steps with knowledge and
commitment from stakeholders. Planning
for disaster recovery takes a determined project planning effort. If you can’t
make out a project plan, can’t write down the dependencies and dates when they
need to be done, they you probably shouldn’t embark on DR until you can do the
project management needed.
For project management, you don’t
need an expert. There are people
certified in project management. The PMI
institute is one place to look for training and advice on how to train people
to become project managers.
In closing, we state the obvious,
that fighting for funding may be the biggest challenge in making a disaster
recovery system come to exist.
Operational expenses do exist when trying to duplicate any service, and
like anything that is not used every day, it needs to be tested now and then. This all adds up to operational expense. The
ability to recover your IT or communications systems from a disaster takes a
large investment in time, training, vendor selection and PATIENCE!
|
The take away lesson from this:
|
Disaster recovery solution
implementation has to come from the top, where the money is distributed and
reallocated. Initial expenditures and
operational costs must be taken into account and put in the budget for a system
to work reliably year after year.
Authentication
= Smart cards.
History shows us many examples of civilized people creating
complete chaos because of lack of authentication. Forged letters, redirected orders, and faked
treasures are woven into the timeline of civilization. How do you ensure authentication?
One way to help ensure a computer user is actually who they
say they are, is to use two factor authentication. Actually, this applies to someone using any
device, whether it is a phone, a military tank, or anything else. Having two factors means you need two things.
The two things could be a smart card, and a password. Or it could be specific computer and a PIN
code, or the duel keys and procedures used to launch a nuclear missile. Any two things. . .thereby increasing the
things you need to authenticate yourself or others.
Why do we need to have authentication? Is this going to be another thing to break
and cause problems?
For one, documents are going digital. Authentication is valuable in my opinion
because we need to validate who someone is that last did something. We need to know who last created a criminal
report, or that officer Bob actually wrote up the murder report.
Maybe you have heard that we all need to start encrypting
our documents, which is true. But how do
you KNOW that a certain person encrypted those documents? Strong authentication helps prove that user X
did a certain action.
More agencies, corporations, and public organizations are
making the slow (and sometimes painful) move to smart card authentication.
What are some of the benefits of smart card authentication
for law enforcement types?
- Decreased
chance of password theft from spyware, and loaning passwords.
- Devices
that are more secure from theft and misuses, if properly implemented!
- System
integrity. That includes less
chance of mistakes by system administrators and “super users”.
- Users
may actually have less problems remembering passwords, as the random one
time passwords generated by the keyfobs (smartcards) may replace passwords
they normally had to remember.
Implementation of smart cards
The roll-out and adaptation of
smart card technology is usually the barrier to entry. Initial cost of the hardware and software to
use the smart cards is relatively low for the more economical cards. Costs rise quickly when you add up the
training and support costs if not implemented effectively. Pilot studies and user acceptance testing is
crucial to gauge acceptance of the technology.
|
The take away lesson from this:
|
Smart cards are just another way of accessing resources, usually
not increasing or decreasing the effort required to authenticate. Enforcing smart card use will increase the
security of your computer systems.
The value of powerful lighting is shown here in a house fire
in Calimesa, California.
Rescue vehicle has a boom light that illuminates the entire scene,
extending 15 feet above the vehicle.
This combined with small telescoping lights from another truck fill in
the shadows with light. The result is a
well lit fire scene, with less accidents and faster operations. Operations in
the dark are no fun! Photo copyright
2006 Rod Deluhery. “A PICTURE IS WORTH A
THOUSAND WORDS”
Product focus: DataPath Portable satellite systems
If instant data communication for a disaster, nothing is
more reliable than satellite providers.
Satellites provide an infrastructure that will almost always be
available even when every other local infrastructure system is damaged. Duluth,
Georgia based
DataPath Inc. supplies quick solutions on a trailer mounted system called the
DataPath 3000. Providing IP-based
network connections, network data connections to the internet are easy.
For a remote area, the DataPath ET2000 weighs much less and
is broken into several cases for transport and can but put together in an
hour. National or international
organizations that deal with emergencies, crisis management of any type should
look at how these satellite systems can fit into the operations of the
organization. Ease of use and simplicity
are important for people needing basic phone and internet service. More advanced users and large groups will
need to look at customizing your specific equipment configurations, and then
testing those devices and configs out in the field.
Technology focus:
Mesh radio.
By the Radio Man
Mesh radio systems is a highly acclaimed technology for
moving data over a mesh of radio devices.
Mesh networks are nothing new, but with the promise of wireless, mesh
pledges to save you money on access points and to extend your range of any such
radio devices. Mesh radio systems are
made up of a broad randge of digital
communication systems. Data networking
in the 802.11 space include products brand names like Mesh Networks (now
Motorola) and Tropos. All these promise
high speed, and a highly reliable connection.
First, I think the most value you get from Mesh is its
reliability. Think of mesh radio as a
way of having multiple paths to get data back to a point. Mesh routing technology looks at costs of
various routes along the mesh and decides the best path. That is the good part.
Now the bad part. My
question on any mesh product is how much latency is created as the data goes
from hop to hop. Mesh is a great thing and can extend high speed RF networks
much farther than normally possible. In my opinion, mesh networks will get
faster and better as the technology matures and more people use them.
As you are all aware, most radio is half duplex, so the radio can only be
either transmitting or receiving, but not both. This half duplex nature
of radio creates small amounts of latency as data is "piped" through
each leg of the mesh. Each radio node may "wait" a few
milliseconds (or longer) for the radio channel to be clear to transmit.
The more hops, the more wait (and degradation) in performance and higher
latency.
Software can only help so much when there are physical limits with the amount
of data that can be moved at one time. Software and protocols can keep
the load balanced and attempt quality of service
only to the limits of hardware.
|
The take away lesson from this:
|
You need to understand how mesh can affect throughput, as
you may get fast response time when going through one or two hops. Past two
hops, speed and latency may be affected.
Facts
on chemical spills.
I personally learned a few years ago the importance the
emergency community puts on chemicals in the environment. I had found some barrels of what looked like
dry cleaning fluid that seemed to have been dumped in a field, behind my
fathers house. I took matters into my
own hands, calling the local fire department, where a guy told me in a calm
voice said “that they would send someone to look at it, when they had some time”
and didn’t seem like it was something they would do that day. I figured they may or may not ever come out
here. I ended up going to a restaurant
about a hour later, and then about two hours later found three fire trucks at
my front door. The captain walked up to
my car.
“Sir, you can’t just call in a hazardous material release
and then leave. We have been waiting
here quite a while for you.”, he said.
“I wasn’t sure you would even come.” I replied, shocked they
where now really concerned, and had been waiting for me for 45 minutes! And over the phone they didn’t seem to
interested.
“Please show us where the chemicals are.” the captain said.
As the hazmat team came out to examine the abandoned waste,
I walked back glad that they did respond to the hazardous waste. I later learned they have the same response
whether it’s one gallon of chemicals left in the street of 55 gallons in a
abandoned field. And it makes sense to
do so. With that, let’s look at some of
the procedures are for chemical spills.
For large spills, the federal government request you report
them to the National
Response Center. The phone number is 1-800-424-8802. For large spills, the federal government may
possibly re-imburse the local agency.
Hazmat teams are the blood hounds of the emergency sector,
sniffing and analyzing liquids, dusts and vapors to identify danger. Hazmat response team members are always
educated on most all possible chemical emergencies. The more chemicals produced
and stored in a regional area would justify better hazmat response.
Most hazmat team members have completed the Hazardous
Materials Operations Level Certification and the Technician Level Certification
for a minimum of 120 hours of training. Expect to find some members have also
completed advanced level certifications, including the Chemistry of Hazardous
Materials certification for a minimum of 240 hours.
The mega merger.
Commentary by Rod D.
You may have read about the proposed merger of AT&T and
Bellsouth. As a quick refresh, here are
some facts:
- There
are many opponents of the deal from internet “net-neutrality” proponents
to plain freedom of choice consumers.
- There
are legal challenges against the effort by consumer groups.
- AT&T's offer to entice the deal, includes
$10-a-month DSL broadband service for new customers for 30 months, and
free DSL modems for new broadband subscribers next year.
- The telecom deal would create a giant with 67 million
phone lines and 57 million customers of Cingular Wireless.
- The
electronic frontier foundation alleges that AT&T has violated the
privacy of its customers and the people they call and email, as well as
broken longstanding communications privacy laws by allowing surveillance
by the NSA.
Why does this matter to law enforcement or emergency
operations? What will be the effects if
the merger is completed? Is it good to
have a monopoly on phone/internet/wireless/cellular services? What
Issue
#3 
