Path to Penetration Tester

Path to Penetration Tester

My path to penetration tester is definitely not a straight line. Looking back, it’s circular. In a way, I’m doing what I was curious about twenty years ago near the beginning of my adult life. I often hear how pen testing is among the most “sexy” or “elite” jobs in infosec. It definitely has it’s ups and downs. Nothing beats the rush of popping shells or owning a Windows domain. At the same time, it has its challenges like finicky clients and long, endless hours of failure between those wins.

Into my later years of high school (late 90’s), administration and I came to an agreement. We all could make each other’s lives more difficult… or less. I made the latter choice. My curiosity about how computer systems and machines worked, and how people interacted with them, was strong. I was frustrated why many people felt there was so little you could do with these beige boxes, so constricted and worked to understand why or what forces were at play. I had a solid moral compass, yet my rebellious attitude kept me in a gray area. Here, I learned that some rules can be bent and some can be broken.

Tip: Attacking systems that aren’t yours or where you don’t have permission is illegal. People who don’t understand your passions are highly detrimental to your freedom. Find an outlet where you can practice like your own lab, capture the flag events, or bug bounties.

College opened my eyes to what the “real” internet was. It was not the cleansed and curated online services like AOL or Prodigy had. 10+ megabits per second was really fast compared to dial-up! I spent less time causing trouble and more time failing calculus. Twice. I also learned to be tolerant to others’ opinions, something that sometimes lacks in forums and discussions around the internet. I still appreciate the academic atmosphere and consider myself a lifelong learner.

Tip: Information security constantly changes. It isn’t a field where you can train, learn it all, then do the work until you retire. It is constantly evolving, and you’ll need to bring your curiosity to this every day.

For the next few years, starting from the early 2000’s, I worked and played with technology in one way or another. The path wasn’t all computer-centered nor security related until years later. I dabbled with some computer work, but I didn’t like the “IT helpdesk” or “IT technician” culture. I really had no idea what I wanted to do with my life, but I did like working for Verizon. The culture was professional, inviting, and I met a lot of really great people along the way. I stumbled upon this certification called Certified Ethical Hacker and thought “hmm, that sounds interesting, I’m busy right now but I should look into that sometime.”

I regret not having much exposure to software development, though something else made up for this in a big way. In 2004, I landed a job as a cell technician with Verizon. It was here that I learned really solid troubleshooting skills, managing time unsupervised, prioritizing tasks, how lousy software makes your head hurt, how software gets better over time, how to automate many things, and how different systems work together (and sometimes against each other). There were many adventures of chasing hurricanes, driving in the middle of the night, in the middle of nowhere to get a cell site back up and running. It was a rush to watch equipment come back to life and their green lights light up.

Tip: There is NOTHING you can’t solve. I spent many long days and nights, facing seemingly insurmountable challenges, with that sinking feeling of utter hopelessness “how will I ever fix this” staring at equipment with red lights (not a good sign). After a few years of this I realized something monumental: there is nothing we individually and especially with the support from a great team, can’t solve. Given enough time, thought, sweat, or support. Nothing. To put it another way, the slogan “try harder” is absolutely true.

I began finding a niche for myself. During one meeting with my manager, I asked “where do you see the industry going in three to five years?” He answered: more data, high speeds, and more power in our pocket. This implied people would use data on their phones more than voice or text. Imagine that? I ran with it and starting thinking about what supports this growth: the networks. At the time, we were in the “3G” era with data speeds of around 1-3 megabits per second on a wireless network. It wasn’t bad, and I was hearing about this new “4G” technology that was supposed to be coming out in the next few years. 50Mbps or more? Unbelievable!

Tip: Keep asking visionaries, mentors, and respected people in your life “where do you see this in 3 to 5 years?” This helps anticipate “waves” so you can position yourslef before they arrive. I think the 3-5 year outlook coincides well with normal business cycles. 1 to 2 years may not be enough time for you to prepare. Beyond 5 years, and the outlook is too distant to predict.

With this mindset, I got to work learning more about the Cisco gear we already had at the cell sites, the T1 lines and future technologies between the sites and the MTSO, and networking overall. The MTSO (or mobile telephone switching office, pronounced “mit’ – so”) was the central office where all of our team’s cell site connections converged and where we connected to other MTSOs and the outside world. In less than a year, I had earned a Cisco CCNA certification. It was a big deal because I was the first in the team and set a huge precedent for others.

Tip: All of this foundational knowledge – IT, networking, time management, soft skills – are crucial in the information security field. In infosec, organization continue to fail at the basics. Don’t discount this.

In 2009, I felt I had a good grasp of what I was doing as a cell tech and wanted to step up to the next level. I got out of the field and in the MTSO. The challenge level jumped dramatically, there was so much more to learn, and I loved it. I had this fantasy that I would be a SuperTech and learn everything about everything. Reality soon proved that was impossible, nor was it interesting – my focus was on networking rather than keeping some legacy systems alive. The exposure to Unix, Linux, large-scale servers, and “big iron” networking gear was incredibly valuable. Try not to think too much about the fact that some Cisco CRS-16 line cards costs more than your house!

Tip: When you feel the anxiety of leaving your comfort zone, being asked to step up into a challenging role: do it! When an experienced leader sees something in you, they know you can succeed, you may not know it yet. Embrace the feeling, be humble, maintain the support system around you, and remember you can solve anything! We become comfortable with the familiarity of the everyday and fail to push ourselves. So, feel the fear and do it anyway!

I was in a Unix class where I asked the instructor “where do you see all of this going in 3 to 5 years?” His answer: security. I was already comfortable with some CCTV work, cameras and DVRs, and badge access systems. I hadn’t yet though about information security as a career path or how it’s far more than fences and surveillance cameras. Interesting…

I started paying attention to security-related news and studying the basics – like Security+ and CISSP, not necessarily to pursue the certifications, but to learn what the field was about and find my place in it. The path looked promising, so I dove in and devoured everything I could: I read, followed the news, blogs, Twitter, listened to podcasts, and studied constantly.

The infosec field, as I quickly realized, is much larger than it looks from an outside perspective. I was aware of penetration testing, yet felt that it was a bit out of my reach. That might take a few more years to get into and I right now was hungry for something – anything – and I was willing to aim low and work my way up. There was a lot to do and I really didn’t know where to start.

Tip: Find a mentor to help you navigate the security field, or any, that you’re interested in getting into. Especially if it’s very different than what your normal day job is. Also look for local groups like ISSA, a local Def Con (not the major conference), or a 2600 group. They can quickly help you meet people who are or in the field or want to get in.

Let the hacking begin At Verizon, I wanted to create something from the bottom up. I saw a general lack of thought around security and wanted to raise everyone’s knowledge to the next level. An apathy that I noticed later, at many companies, was “security is someone else’s job.”

I did learn and teach others quite a bit about threat hunting and threat modeling (before these were terms), proactively looking for issues, logging, configuration management, automating anything I could, and monitoring what a “normal” network looks like so we can find the outliers quickly. These were great skills for any network administrator, let alone a security pro, and it was something we hadn’t done up to this point.

I started networking with others because I wanted to create group to do this sort of work. It was a bit much to add yet another task to the general technicians. It also seemed like a heavy lift for many I spoke to, who were often too busy or too insecure to help me out. Trying to create something from nothing was an uphill battle and in a large company, things often don’t work this way.

I soon realized that I should join an already established team – which plenty existed in the company, though all seemed siloed and nobody talked to one another. A communication company where other groups don’t talk to one another? Yeah, that’s the running joke internally. The tip that anything is achievable was still valid here, though not always in a realistic timeline. Pick your battles.

I took a methodical approach to getting into a security position. I set out to learn what skills various job openings were looking for. I set up an alert in the internal job search system for the word “security.” There was at least one hit per week, and typically one per day. Often, they included the word security as an associated skill: “brew coffee and securely shred paper.” A minority were security focused; the ones I was after. The main goal here was to learn what skills were in demand so I could figure out how to best fit or learn them. To eventually pivot into one of these positions.

Tip: hack the job search. Rather than “here are the skills I have, what can I do with them?” Think “here are the jobs that look interesting, and those are the skills they’re looking for. What do I need to learn to get there? What patterns or commonalities exist?” If you have the luxury of time, it’s a good method.

This turned into spreadsheets analyzing content and keywords, names, tools mentioned, re-postings (for those not being filled), and other useful bits of information from the postings. I heard the term “big data” by this point, and I often found myself pretty good at analyzing huge datasets and pulling something interesting out of them. This continued for six months. By looking at the manager’s org chart and seeing each person’s work location I could quickly find those who were centralized in one office versus distributed throughout the country. Working remote wasn’t the norm in those days, yet the company was embracing the practice and it was on the rise.

I found a job opening for an internal application security specialist. It looked interesting, and I applied for it. I had an interview with the hiring manager, who seemed personable enough, though was a little apprehensive about someone working remote (they were in NJ). I then had a “technical” interview with one of his staff. He was very intelligent, and knew the subject matter well. I, however stumbled through parts of this interview and felt I did mediocre.

It turns out there was some politics behind the scenes. The hiring manager had recently been promoted, and replaced the former manager. He was (re)building this team. Apparently, the former manager took a few of his people when he left. Red flags! He strung me along for a few weeks, saying he wanted to hire me, that I was one of the “finalists” and he was getting approvals or whatever. I didn’t get that job, which I later learned was a good thing.

The hunt continued. I found a group in corporate security. I had some concerns about the culture of my familiar line of business (Verizon Wireless) being very different to the “Corporate” side. I did like the idea of its centrality – it touched all parts of the company. This also wasn’t some small pocket of security outlaws doing their own thing – it seemed more esteemed and legitimate. I reached out to the hiring manager, sent them a resume, and opened a conversation about the job and what I could bring to the table. I think most people simply apply for the position and hope for the best, I at least wanted to apply “warm” after already introducing myself. The manager over this group responded very quickly and was receptive, a good sign. The strategy was also to avoid the dreaded and somewhat nebulous applicant management system (AMS). I knew they might filter applicants who didn’t meet a certain (unknown) criteria, keywords, or whatever. Some applicants weren’t getting their names to hiring managers for mysterious reasons. Another hack!

I applied for this one and soon had interviews lined up. The first, in typical fashion, was with the hiring manager. She and I synced up well and I respected that she had many years of leadership experience. The second “technical interview” was with two of her staffers, both incredibly experienced and whom I highly respect to this day. This interview was surreal, it felt more like a conversation. One of the interviewers said early on “I’m done, that’s all I need to know.” He was a techie at heart, so we easily found common talking points. I took the opportunity to ask them about themselves and their work. If I wasn’t going to land the job, at least I would tap everyone I came in contact with for some experience and perspective.

Tip: even if you don’t get the job, strive to learn something from the people doing it or hiring for it. What does a successful candidate look like? What are the struggles and the wins? What I do to better prepare for a position like this if it becomes available in the future? Those questions demonstrate tenacity – something you’ll need in infosec.

I was offered the job and was super excited to hit the ground running in my first security role! I later talk to one of the technical interviewers. I mentioned that the interview felt more like a conversation and was curious why he wanted to cut it short. He mentioned that he could sense my knowledge and confidence of the subject, and the ability to keep up with the topics. That’s why it felt more like a conversation, because we were speaking to each other as nearly equals.

I met a peer who was also hired into this group at around the same time. He personally knew the hiring manager for that job that I didn’t get before this one. It turns out, I dodged a MASSIVE bullet! So what seemed like a failure was a positive in disguise.

Corporate security was a challenge that combined learning and working around a bureaucracy, working with incredibly limited resources, and influencing people. It was an incredible opportunity to learn the business of security. This group generally focused on third-party risk; basically, working on securing the supply chain. There was also an occasional side-project like a chatty chatbot, analyzing logs, and employees with inappropriate software on their computer. It was less technical than what I had done in the past, more legal-related, and no two days were alike.

These people loved their work, they were passionate, brilliantly intelligent, and were (for the most part) great at what they did given the limited tools and circumstances we had to work with. Though it was a distributed group, we kept a strong inclusion and supportive attitude from everyone. The leadership, whether they strived for it or not, built a really solid culture for this team.

The “business of security” is a challenge rooted in convincing people why security is a good idea, because someone has to pay for it. Paying for something that prevents loss is very different than paying for something that has a more defined return on investment. This ignores the fact there are often contractual requirements to meet a level of security. It required learning the real depth of risk and managing it well. These conversations required heaps of empathy and understanding, building relationships, and skillfully navigating these conversations. When I started, I wasn’t very comfortable or experienced running conference calls and meetings. After some time, it became second nature.

Tip: The technical skills you learn will become obsolete. The soft skills you learn will last your entire career.

Tip: Start meetings on time, stay on-topic, don’t let anyone hijack your meeting, and importantly end it on time.

I experienced some dark times as well. I often left the office feeling mentally exhausted, with a headache, and often in a bad mood. I was getting burned out. The combination of the shortage of people, the high demands, fast pace, and the fairly new and dynamic type of work makes it a fairly common issue in infosec. I eventually found my triggers: multitasking and not taking a break throughout the day. I learned a lot about human multitasking, science has proven we don’t do it well. To my other detriment, I can focus on something for hours.

Tip: take care of yourself and your health. It is never worth sacrificing this for your job.

Tip: Create “fake” meetings in your calendar as uninterrupted time for deep work. This time is sacred and you have to protect it.

Tip: Look into Steven Covey’s “Four Quadrants” time management system. Learn the art of respectfully saying “no.” The result is don’t won’t have to prioritize things you decline!

Tip: Don’t let yours or others’ emotions drive your response under stressful times. When things are hitting the fan, we can all lose our sensibilities. Step back, pause, and take the time to think clearly about whatever situation you’re facing. It seems counterintuitive, but it can help when everyone’s losing their mind.

I really missed technical work. Technology seems so much simpler and predictable than humans. I cautiously started back on the job search. Ideally, it would really be nice to do something in security that didn’t involve nagging people about their lack of it. At this point, I had worked in various groups in Verizon for about 18 years. Then a hit from my search showed up: a job opening for a penetration tester. This was a team working for external clients, not internal. There were benefits to external facing work: a revenue-generating group tends to have more flexibility with buying tools, training, and supplies to improve themselves and their work. I went for it.

As before, the interview was multi-staged with the hiring manager followed by a more technical interview. This manager, refreshingly, was definitely knowledgeable about the technical aspects of penetration testing. This was a bit out of the ordinary as most people in management don’t get the opportunity to get into the technical work. He had worked as a penetration tester for the last 10+ years and was highly skilled. He was a seasoned manager who built and led this team for a number of years. I liked his vision and experience on this team.

I was honest that I knew my way around networks and computer systems well. Windows and Unix/Linux were second-nature and years spent troubleshooting the depths of TCP/IP with tools like Wireshark and the overall Cisco knowledge in the years of working in Network were all going for me. Yet, I admitted there was still a LOT I would need to learn to attack these things. I was eager to learn and he took a chance on me. I’m incredibly grateful for the opportunity.

Tip: If you see a job opening where you don’t fully meet the requirements yet it’s something you’re passionate about, go for it anyway. Be honest about your skill shortcomings, if you’re not then it will come back to haunt you. I notice a slow shift in hiring practices: people who are very teachable versus those who may be fully qualified or experienced (potentially stubbornly stuck in their ways) may be the ones selected for a job. Be the person who has the eagerness to learn, a great attitude, and gets along with others.

I did get the job. It does have it’s ups and downs. The challenge level is high, and not a day goes by that I don’t learn something new, just how I like it!

In a later article, I’ll talk about those ups and down and the daily work. Stay tuned…

Disclaimer: these views are my own and not that of my employer.