Sudonix

Sudonix

Any improvement on Ghost addon?

Solved Configure

Did this solution help you?
Did you find the suggested solution useful? Why not buy me a coffee? It's a nice gesture, and a great way to show your appreciation💗

  • phenomlabundefined

    Technical issue? You might be overthinking it

    Blog
    1
    +0
    0 Votes
    1 Posts
    157 Views
    phenomlabundefined

    Once in every while, you encounter a repetitive issue that no matter what you try to do to resolve it, the problem manifests itself over and over again - sometimes, even on a daily basis. Much of how the issue is remediated really depends on the person assigned to the task.

    You might be puzzled at why I’d write about something like this, but it’s a situation I see constantly - one I like to refer to as “over thinker syndrome”. What do I mean by this ? Here’s the theory. Some people are very analytical when it comes to problem solving. Couple that with technical knowledge and you could land up with a situation where something relatively simple gets blown out of all proportion because the scenario played out in the mind is often much further from reality than you’d expect. And the technical reasoning is usually always to blame. Sometime around 2007, a colleague noticed that the Exchange Server (2003 wouldn’t you know) would suddenly reboot half way through a backup job. Rightly so, he wanted to investigate and asked me if this would be ok. Anyone with an ounce of experience knows that functional backups are critical in the event of a disaster - none more so than I - obviously, I have the go ahead. One bright spark in my team suggested a reboot of the server, which immediately prompted the response

    “…it’s rebooting itself every day, so how will that help ?”

    The investigation

    Joking aside, we’ve all heard the “have you rebooted” question touted at some point during helpdesk discussions, but this one was different. A system rebooting itself is usually symptomatic of an underlying issue somewhere, and my team member was ready for the task ahead. Stepping up to the plate, he asked if it was ok to install some monitoring software on the server. Usually, installing additional software components in a production server without testing first is a non-starter, but seeing as we needed to get this resolved as quickly as possible to reinstate the nightly backup (which incidentally hasn’t run successfully for 3 days by now), I provided approval to proceed without question. There’s a leap of faith at this point, as you could cause more problems than that you actually set out to resolve in the first place, but, as with anything related to information technology, someone’s you have to accept an element of risk. The software itself was actually for the RAID controller and motherboard  The assigned technician had already decided it was related to something along the lines of a faulty RAM module, or perhaps an issue with the controller itself. My thoughts leaned elsewhere already at this point - is the server reboots itself at exactly the same time every day then there is an established pattern which should be investigated first. It’s a logical approach, but it’s a common trait for technical support staff to sometimes think outside of the box - or in this case, outside of the building. Not wanting to push my opinion, or trample on anyone’s toes, I decided to remain quiet and see just how far this would go before intervention was required.

    In this case, not very far. The following morning after another unannounced nightly reboot, the error “the previous shutdown at [insert time and date here] was unexpected” showed up in the event log. No real surprises there, and once again, exactly the same time as the previous night. I asked my technician for an update, and he informed me that he believed that the memory was faulty and somehow causing the server to blue screen and reboot. That was actually a reasonable response and so I commended him on his research and findings, but also reminded him to perform a manual backup so that we at least had something to revert to in the event of a failure. Later that afternoon, the same tech approached me and said that he had ordered some replacement memory, and wanted to arrange downtime to fit it. Trying to keep a poker face and remain passive, I agreed and the memory was replaced the same evening around 10pm. At 2am the following morning, kaboom ! - the server rebooted itself again. Not wanting to admit defeat, our courageous tech suggested that the problem could be due to the system overheating. Another fair point, but not realistic as you’d see this in event log as a thermal shutdown. I willingly entertained this, and allowed investigations into the CPU temperature to begin - after another manual backup. Unsurprisingly, the temperature data returned no smoking gun, so that was abandoned. The next port of call was to reapply the service pack. Now, I’ll admit that this used to fix a multitude of issues under Windows NT Server (particularly Service Pack 4) but not under Windows 2003. I declined this for obvious reasons - if you reapply the service pack, you run the risk of overwriting key DLL files that could (and often will) render Exchange inoperable. Not being prepared to introduce an unprecedented risk into what was already becoming something of a showcase, I suggested that we look elsewhere.

    The exasperation

    The final (and honestly more realistic suggestion) was to enable verbose logging in Exchange. This is actually a good idea, but only if you suspect that the information store could be the issue. Given the evidence, I wasn’t convinced. If there was corruption in the store, or on any of the disks, this would show itself randomly through the day and wouldn’t wait until 2am in the morning. Not wanting to come across as condescending, I agreed, but at the same time, set a deadline to escalation. I wasn’t overly concerned about the backups as these were being completed manually each day whilst the investigations were taking place. Neither was I concerned at what could be seen at this point as wasting someone’s time when you think you may have the answer to what now seemed to be an impossible problem. This is where experience will eclipse any formal qualifications hands down. Those with university degrees may scoff at this, but those with substantially analytical thinking patterns seem to avoid logic like the plague and go off on a wild tangent looking for a dramatically technical explanation and solution to a problem when it’s much simpler than you’d expect. Hence the title of this article - Avoid the “bulldozer to find a china cup” scenario. After witnessing another pained expression on the face of my now exasperated and exhausted tech, I said “let’s get a coffee”. In agreement, he followed me to the kitchen and then asked me what I thought the problem could be. I said that if he wanted my advice, it would be to step back and look at this problem from a logical angle rather than technical. The confused look I received was priceless - the guy must have really though I’d lost the plot. After what seemed like an eternity (although in reality only a few seconds) he asked me what I meant by this. “Come with me”, I said. Finishing his coffee, he diligently followed me to the server room. Once inside, I asked him to show me the Exchange Server. Puzzled, he correctly pointed out the exact machine. I then asked him to trace the power cables and tell me where they went.

    As with most server rooms, locating and identifying cables can be a bit of a challenge after equipment has been added and removed, so this took a little longer than we expected. Eventually, the tech traced the cables back to

    …an old looking UPS that had a red light illuminated at the front like it had been a prop in a Terminator film.

    The realisation

    Suddenly, the real cause of this issue dawned on the tech like a morning sunrise over the Serengeti. The UPS that the Exchange Server was unexpectedly connected to had a faulty battery. The UPS was conducting a self test at 2am each morning, and because the bypass test failed owing to the burnt battery, the connected server lost power and started back up after the offending equipment left bypass mode and went online.

    Where is this going you might ask ?  Here’s the moral of this (particular, and many others like it) story

    Just because a problem involves technology, it doesn’t mean that the answer has to be a complex technical one Logic and common sense has a part to play in all of our lives. Sometimes, it makes more sense just to step back, take a breath, and see something for what it really is before deciding to commit It’s easy to allow technical expertise to cloud your judgement - don’t fall into the trap of using a sledgehammer to break an egg You cannot buy experience - it’s earned, gained, and leaves an indelible mark

    Let’s hear your views. Did you ever come across a situation where no matter what you tried, nothing worked ? Did the solution turn out to be much simpler than you’d have ever thought ?

  • phenomlabundefined

    Performing a migration? Ensure you have a back-out plan

    Blog
    1
    +0
    0 Votes
    1 Posts
    169 Views
    phenomlabundefined

    One of the most important safety nets in IT Operations is contingency. Every migration needs a rollback plan in the event that things don’t quite go the way you’d expect, and with a limited timeline to implement a change, or in some cases, a complete migration, the rollback process is one that is an essential component. Without a plan to revert all changes back to their previous state, your migration is destined for failure from the outset. No matter how confident you are (I’ve yet to meet a project manager who doesn’t build in redundancy or rollback in one form or another) there is always going to be something you’ve missed, or a change that produces undesirable results.

    It is this seemingly innocent change that can have a domino effect on your migration - unless you have access to a replica environment, the result of the change cannot be realistically predicted. Admittedly, it’s a simple enough process to clone virtual machines to test against, but that’s of no consequence if your change relates to those conducted at hardware level. A classic example of this is a firewall migration. Whilst it would be possible to test policies to ensure their functionality meets the requirement of the business, confirming VPN links for example isn’t so straightforward - especially when you need to rely on external vendors to complete their piece of the puzzle before you can continue. Unless you’re deploying technology into a greenfield site, you do not have the luxury of testing a VPN into a production network during business hours. Based on this, you have a couple of choices

    You perform all testing off hours by switching equipment for the replacement, and perform end to end testing. Once you are satisfied everything works as it should, you put everything back the way you found it, then schedule a date for the migration. You configure the firewall using a separate subnet, VLAN, and other associated networking elements meaning the two environments run symmetrically

    But which path is the right one ? Good question. There’s no hard and fast rule to which option you go for - although option 2 is more suited to a phased migration approach whilst option 1 is more aligned to “big bang” - in other words, moving everything at the same time. Option 2 is good for testing, but may not reflect reality as you are not targeting the same configuration. As a side note, I’ve often seen situations where residual configuration from option 1 has been left behind, meaning you either land up with a conflict of sorts, or black hole routing.

    Making use of a rollback

    This is where the rollback plan bridges the gap. If you find yourself in a situation where you either run out of time, or cannot continue owing to physical, logical or external constraints, then you would need to invoke your rollback plan. It’s important to note at this stage that part of the project plan should include a point where the progress is reviewed and assessed, and if necessary, the rollback is executed. My personal preference is within around 40% of the allocated time window - all relevant personnel should reconvene and provide status updates around their areas of responsibility, and give a synopsis of any issues - and be fully prepared to elaborate on these if the need arises. If the responsible manager feels that the project is at risk of overrunning it’s started time frame, or cannot be completed within that window, he or she needs to exercise authority to invoke the rollback plan. When setting the review interval, you should also consider the amount of time required to revert all changes and perform regression testing.

    Rollback provides the ideal opportunity to put everything back how it was before you started on your journey - but it does depend on two major factors. Firstly, you need to allocate a suitable time period for the rollback to be completed within. Secondly, unless you have a list of changes that were made to hardware - inclusive of configuration, patching, and a myriad of others, how can you be sure that you’ve covered everything ?

    Time after time I see the same problem - something gets missed, and turns out to be fundamental on Monday morning when the changes haven’t been cross checked.

    So what should a contingency plan consist of ?

    One surefire way to ensure that configurations are preserved prior to making changes is to create backups of running configs - 2 minutes now can save you 2 days of troubleshooting when you can’t remember which change caused your issue.  For virtual machines, this is typically a snapshot that can be restored later should the need arise. A word to the wise though - don’t leave the machine running on snapshot for too​ long as this can rapidly deplete storage space. It’s not a simple process to recover a crashed VM that has run out of disk space.

    Keep version and change control records up to date - particularly during the migration. Any change that could negatively impact the remainder of the project should be examined and evaluated, and if necessary, removed from the scope of works (provided this is a feasible step - sometimes negating a process is enough to make a project fail)

    Document each step. I can’t stress the importance of this enough. I understand that we all want to get things done in a timely manner, but will you realistically remember all the changes you made in the order they were implemented ?
    Use differential tools to examine and easily highlight changes between two configurations. There are a number of free tools on the internet that do this. If you’re using a Windows environment, a personal favourite of mine is WinMerge. Using a diff tool can separate the wood from the trees quickly, and provides a simple overview of changes - very useful in the small hours, I can assure you.
    Working on a switch or firewall ? Learn how to use the CLI. This is often superior in terms of power and usually contains commands that are not available from the GUI. Using this approach, it’s perfectly feasible to bulk load configuration, and also back it out using the same mechanism.

    What if your rollback plan doesn’t work ? Unfortunately, there is absolutely no way to simulate a rollback during project planning, and this is often made worse by many changes being made at once to multiple systems. It’s not that the rollback doesn’t work - it’s usually always a case of settings being reverted before they should be. In most cases, this has the knock on effect of denying yourself access to a system - and it’s always in a place where there are no local support personnel to assist - at least, not immediately. For every migration I have completed over my career, I’ve always ensured that there is an alternative route to reach a remote device should the primary path become inaccessible. For firewalls, this can be a blessing - particularly as they usually permit access on the public interfaces.

    However, delete a route inadvertently and you are toast - you lose access to the firewall full stop - get out of that one. What would I do in a situation like this where the firewall is located in Asia for example, and you are in London ? Again - contingency. You can’t remove a route on a firewall if it was created automatically by the system. In this case, a VLAN or directly connected interface will create it’s own dynamic route, and should still be available. If dealing with a remote firewall, my suggestion here would be Out Of Band Management (OOBM), but not a device connected directly to the firewall itself, as this presents a security risk if not configured properly. A personal preference is a locally connected laptop in the remote location that uses either independent WiFi or a 3G / Mifi presence. Before the migration starts, establish a WebEx or GoToMeeting session (don’t forget to disable UAC here as that can shoot you in the foot), and arrange for a network cable to be plugged into switch fabric, or directly. Direct is better if you can spare the interface, as it removes potential routing issues. Just configure the NIC on the remote machine with an address in the same subnet add the interface you’re connected to, and you’re golden.

    I’ve used the above as a get out of jail free card on several occasions, and I can assure you it works.

    So what are the takeaways here ?

    The most important aspect is to be ready with a response - effectively a “plan b” when things go wrong. Simple planning in advance can save you having to book a flight, or foot the expense of a local IT support firm with no prior knowledge of your network - there’s the security aspect as well; you’d need to provide the password for the device which immediately invokes a change once the remediation is complete. In summary

    Thoroughly plan each migration and allow time for contingency steps. You may not need them, and if you don’t, then you effectively gain time that could be used elsewhere. Have an alternative way of reaching a remote device, and ensure necessary third party vendors are going to be available during your maintenance window should this be necessary. Take regular config backups of all devices. You don’t necessarily need an expensive tool for this - I actually designed a method to make this work using Linux, a TFTP server, and a custom bash script - let me know if you’d like a copy 🙂 Regularly analyse (automated diff) configuration changes between configurations. Any changes that are undocumented or previously approved are a cause for alarm and should be investigated Ensure that you have adequate documentation, and steps necessary to recover systems in the event of failure

    Any thoughts or questions ? Let me know !

  • phenomlabundefined

    How to destroy a community before it's even built

    Blog
    4
    +0
    5 Votes
    4 Posts
    221 Views
    phenomlabundefined

    @crazycells I guess the worst part for me was the trolling - made so much worse by the fact that the moderators allowed it to continue, insisting that the PeerLyst coming was seeing an example by allowing the community to “self moderate” - such a statement being completely ridiculous, and it wasn’t until someone else other than myself pointed out that all of this toxic activity could in fact be crawled by Google, that they decided to step in and start deleting posts.

    In fact, it reached a boiling point where the CEO herself had to step in and post an article stating their justification for “self moderation” which simply doesn’t work.

    The evidence here speaks for itself.

  • phenomlabundefined

    Recovering from a server disk array failure in 1998

    Blog
    1
    0 Votes
    1 Posts
    123 Views
    phenomlabundefined

    dc1.webp
    Why is it that all outages seem to happen at 5:30pm on a Friday afternoon ? Back in the day during 1998 when DEC (yeah, I’m old - shoot me) was still mainstream and Windows NT Server 4.0 was the latest and greatest, I was working for a commodity trading firm in the West End as an IT Manager. The week had typically gone by with the usual activity - nothing too major to report apart from the odd support issue and the usual plethora of invoices that needed to be approved. Suddenly, one of my team emerged from the comms room and informed me that they had spotted a red light on one of the disks sitting in the Exchange server. I asked which disk it was, and said we’d need to get a replacement.

    For those who haven’t been in this industry for a years (unlike me) DEC (Digital Equipment Corporation) was a major player in previous years, but around 1998 started to struggle - it was then acquired by Compaq (who later on down the line in 2002 were acquired themselves by Hewlett Packard). This server was a beast - a DEC server 5000 the size of an under the counter fridge with a Mylex DAC960 RAID controller. It was so large, it had wheels with brakes. And, like a washing machine, was incredibly heavy. I’m sure the factory that manufactured servers in the 90’s used to pour concrete in them just for a bit of fun…

    Here’s a little glimpse for nostalgia purposes
    decserver5000.webp
    Those who remember DEC and it’s associated Mylex DAC960 RAID controller will also recall that the RAID5 incarnation was less than flawless. In modern RAID deployments, if a disk was marked as faulty or defunct, the controller would effectively blacklist the disk meaning that if it were to be removed then reinserted, any bad blocks would not be copied into the array hence causing corruption - it would be rejected.

    Well, that’s how modern controllers work. Unfortunately, the DAC960 controller was one of those boards that when coupled with old firmware and the NT operating system created the perfect storm. It was relatively well documented at the time that plugging a faulty drive back into an array could cause corruption and spell disaster. My enterprising team member had spotted the red light on the drive, then decided to eject it out of the array. For some unknown reason, instead of taking it back to his desk to order a replacement, he reinserted the it back into the array. Now, for those of you that actually remember the disks that went inside a DEC server 5000, you’ll know that these things were like bricks in plastic containers. They were around 3 inches in height, about 6 inches long, and quite heavy. These drives even had a eject clip on each side meaning that you had to press both sides of the disk carrier and then slide out the drive before it could be fully removed. Inserting a replacement drive required much the same effort (except in reverse), and provided a satisfyingly secure “clunk” as the interface of the drive made contact with the RAID controller bus.

    No sooner had I said the words

    “…please tell me you didn’t plug that disk back in……”

    to my team member, our central helpdesk number lit up like a Christmas tree in Times Square with users complaining they couldn’t get into email. I literally ran into the comms room and found the server with all drive bays lit solidly as if suspended in its own cryogenic state. For sake of schematics, a standard RAID5 configuration looks like the below. Essentially, the “p” component is parity. This is the stripe that contains information about the array and is spread across all disks that are members. In the event that one fails, the data is still held across the remaining drives, meaning still accessible - with a reduction in performance. The data is written across the disks in one write like a stripe (set).
    raid5_ok.webp
    At this point I’d already realised that the array had been corrupted by the returning faulty disk, and the bad stripe information was now resident on all the remaining drives. Those who understand RAID will know that if one drive in a RAID5 set fails, you still have the other remaining drives as a resilient array - but not if they are all corrupted. What I am alluding to here is shown below. The stripe was now unreadable, therefore, none of the disks were accessible
    raid5_broken.png.webp
    The server had completely frozen up and would not respond. I’m no fan of force powering a server off in the best of circumstances, but what choice did we have ?

    The server was powered off, then turned back on again. I really was hoping that this was just a system freeze and a reboot would make all our problems go away. The less naïve and experienced part of me dragged my legs towards the backup storage area (yes, we had a rotation pool of 2 weeks on site and 2 weeks off), and started collecting the previous day’s backup from the safe. As it stands, this was clearly the next logical step. Upon restart, we were met with the below shortly after NTOSKERNEL completed it’s checks
    bsod.webp
    (Not the actual BSOD of course - camera phones didn’t exist in 1998 - but as close as it gets)

    Anyone familiar with the Windows operating system will have bumped into this at some point in their career, and by the more commonplace acronym BSOD (Blue Screen Of Death). Either way, it’s never a good sign when you are trying to recover a system. One of the best messages displayed by a BSOD is

    IRQL_NOT_LESS_OR_EQUAL

    I say “best” with a hint of sarcasm of course as this message is completely useless and doesn’t mean anything to anyone as such. As the internet back in 1998 was fairly infantile, gaining a decent insight wasn’t as simple or clear cut as it is today. Looking at the problem from a sensible angle, it was fairly obvious that the DAC960 controller had either failed completely, or couldn’t read the disks and caused the crash. Deciding not to invest too much time in getting this system back to life, I fired up it’s dormant sister (yes, we had two fridges :)) which started with no issues. This secondary server was originally purchased to split the load of the mailboxes across two servers for resilience purposes, but never came to fruition owing to a backlog of other projects that were further up the chain of importance. Had this exercise have taken place, only 50% of the office would have been impacted - typical.

    With the server started, we then began the process of installing Exchange. Don’t get too excited - this was Exchange 5.5 and didn’t have any formal link to Active Directory, so it was never going to be the case of installing Exchange in disaster recovery mode, then playing back the database. Nope. This was going to be a directory restore first, followed by the Information Store.

    With Exchange installed and the previous service packs and hotfixes applied (early versions of Exchange had a habit of not working at all after a restore unless the patching​ level was the same), BackupExec 6.2 (yes, I know) was set to restore to an alternative Exchange server, and the tapes loaded into the robotic arm cradle. In hindsight, it would have been a better option to install BackupExec on the Exchange server itself, and connect the tape drive to the SCSI connector. However, can you find a cable when you really need one ? In any case, the server was SCSI2 when the loader was SCSI1. This should have set alarm bells ringing at the time, but with the restore started, we went back to our seats - I then began the task of explaining to senior management about the cause of the outage and what we were doing to resolve the problem. As anyone with experience of Microsoft systems knows, attempting to predict the time to restore or copy anything (especially back in the 90’s) wasn’t a simple task, as Windows had a habit of either exaggerating the time, or sitting there not responding for ages.

    Rather like a 90’s Wikipedia, NT wasn’t known for it’s accuracy.

    I called home and solemnly declared I was in for a long night. It’s never easy explaining the reasons why or attempting to justify the reasons you need to work late to family members, but that’s another story. Checking on the progress of the restore, we were averaging speeds of around 2Mbps ! Cast your mind back to 1998 and think of the surrounding technology. Back in the (not so good) old days, modern switching technology and 10Gbps networks were non existent. We were stuck with old 3Com 10Mbps hubs and an even slower Frame Relay connection (256k with 128k ISDN backup) as the gateway. To make matters worse, our internet connection was based on dialup technology using a SHIVA LanRoverE. Forget 1Gb fibre - this thing dished out an awesome [sic] 33.6k speed or even 56k if you were using ISDN. Web Pages loading in about 20 seconds was commonplace - downloading drivers was an absolute nightmare as you can imagine.

    Back to the restore. Having performed the basic math, and given the size of the databases (around 70Gb on a DLT 40 that was compressed to 80Gb), this was going to take over 24 hours. If you think about how hubs used to work, this meant that the 10Mbps speed of the device was actually shared across all 24 ports. This effectively reduces the port speed to 0.42Mbps - and that really depends on what the other ports are doing at the time. The restore rate remained at around 2Mbps for hours, and rather than everyone sit there watching water evaporate, I sent home the remaining staff and told them to be on standby for the entire weekend. I really couldn’t stomach food at this point, and ended up working into the night on other open tasks in an effort to catch up. I ended up falling asleep at my desk around 2am, and then being woken by the sound of my mobile (a Nokia of course) ringing. Looking at the clock, it was 5am. Checking the restore, it had progressed to the information store itself and was around 60% completed. After another 15 hours in the office, the restore finally completed.

    Having restarted all of the Exchange services, even the information store came up, which really was good news. However, browsing through the mailboxes I noticed that only a quarter of the 250+ I was expecting were listed. Not knowing much about the Exchange back end at the time, I contacted a so-called Exchange specialist based in Switzerland (in case you’re wondering, we were a Swiss headquartered entity, and all external support came from there). This Exchange specialist informed me that the backup hadn’t completed properly, and a set of commands needed to be run in BackupExec to resolve this. Of course, this also meant that the restore process had to be restarted - there goes another 24+ hours, I thought to myself. With the new “settings applied” and the restore process restarted, I decided that I wasn’t going to sit in the office for another day waiting for the restore to complete, and so I decided to call one of my team to come in and occupy the watchtower.

    Getting hold of someone was much more difficult than I had imagined. After letting the remainder of the team go, they all forged an exodus to the nearest door like iron filings to a magnet. So much for team ethic I thought. Eventually, I managed to get hold of a colleague who, after much griping, agreed to come into the office. I wouldn’t have minded as much if he didn’t live less than 15 minutes away, but that’s another story. My colleague arrived around 30 minutes later, and then I left the office. Getting home wasn’t a simple task. In the UK, there are often engineering works taking place over the weekend - particularly on the tube, and in most cases, local rail providers also - mine included. What should have taken about 2 hours maximum took 4, and by the time I got home, I flopped into bed exhausted. Needless to say this didn’t go down particularly well with my wife who saw me last on the previous morning - especially as after 3 hours of restlessness and a general inability to sleep, I was called by senior management - and was asked to go back in.

    By now, my already frustrated wife’s temperature went from 36.9c to an erupting volcano equivalent in less than a split second. I fully appreciated her response, but I was young (well, younger), eager to impress, and also had a sense of ownership. After a somewhat heated exchange, I left for the office. I arrived in much the same time as it took me to get home in the first place, and found that the restore was of course still running. My colleague made some half baked excuse that he needed to leave the office as he had a “family emergency”. Not really in the mood to argue this, I let him leave. I then got on a conference call with the consultant we had been using. Unsurprisingly, the topic of the restore time came up.

    “…You have a very slow network…” said the consultant.

    “…No s**t Sherlock…”  I thought. “…Do you honestly think I’m sitting here for my health ? …”

    I politely “agreed”.

    Eventually, the restore process completed. With a sudden feeling of euphoria, I went back into the comms room to start the services and… to my dismay, found once again that only a third of the recipients appeared in the directory. The term “FFS” didn’t go anywhere near being an accurate portrayal of my response. I was brutally upset. Hopelessly crushed. On the verge of losing it… (ok, perhaps that’s overkill). There had to be a reason for this. Something we’d missed, or just didn’t understand. I went looking for answers on a 1998 version of Yahoo (actually, I think it may have been Lycos), and found an article relating to the DS/IS Consistency Adjuster in Exchange 5.5 - this isn’t the exact resource I found, but it goes a long way to describe the fundamental process. The upshot is that the consistency adjuster needed to be run to synchronise the once orphaned mailboxes with the directory service. This entire process took​ a couple of hours - whilst that seems inconceivable to even the extreme Luddite, this is 1998 with SCSI1 drives, a Pentium II Processor, and 512Mb ram.

    After the process completed (which incidentally looked like this)
    dsisadjuster.webp
    I could then see all mailboxes ! After performing several somersaults around the office (just kidding here, but I can tell you I felt like doing it), I confirmed with a 25% random user test that I had access to mailboxes. Unfortunately, I couldn’t see any new mail arriving, but that was only due to a stalled mail connector on the server in Switzerland that received external mail. After a quick reboot of this gateway, mail began to flow. After around an hour of testing, I was happy that everything was working as expected. As for the consultant who had just wasted hours of my life, it’s just as well he wasn’t in the same country as me, let alone room. I went home elated - to an extremely angry wife. She’s since forgiven me of course, and now looking back, I really appreciate why - she was looking out for me, and concerned - I just didn’t appreciate that at the time.

    Come Monday morning, users were back into email with everything working as expected. An emergency Exchange backup had been run, and I was in the process of writing up my postmortem report for senior management. I then got a phone call. Anyone remember a product by Fenstrae called Faxination ? This was peered with Exchange 5.5, and had stopped working since the crash. The head of operations demanded that this was resolved as a priority… Another late night… another argument at home, but that’s a story for another day.

  • phenomlabundefined

    Surface Web, Deep Web, And Dark Web Explained

    Blog
    3
    0 Votes
    3 Posts
    184 Views
    phenomlabundefined

    @justoverclock yes, completely understand that. It’s a haven for criminal gangs and literally everything is on the table. Drugs, weapons, money laundering, cyber attacks for rent, and even murder for hire.

    Nothing it seems is off limits. The dark web is truly a place where the only limitation is the amount you are prepared to spend.

  • phenomlabundefined

    Wasting a scammer's time is hugely entertaining

    Blog
    1
    +0
    1 Votes
    1 Posts
    258 Views
    phenomlabundefined

    What would happen if a cyber criminal attempted to scam a security professional ? Well, some time ago, this happened to me. Like everyone, I certainly receive my fair share of junk email, scams, and pretty much everything else that the internet these days tends to throw at you. For the most part, each one of these “attacks” is ignored. However, one caught my eye after only the first paragraph. Not only was the format used absurd, but the supposedly “formal tone” was nothing short of a complete joke. Unfortunately, there really is no “TL;DR” synopsis for this particular event.

    Scrolling to the bottom of the article is of course up to you, but you’ll not only miss out on key information - you’ll also miss out on my sarcasm 🤣

    Admittedly, this “scam” sounds far fetched. But, believe it or not, this particular campaign has a high success rate (and, all content in this article actually happened). If this were not the case, would a potential criminal go to such lengths to impersonate and engage ? No. They rely on that one human trait known as trust. Trust which in this case is readily exploited. I promise that this article will be worth your while reading. Ready ? Buckle up. its going to be an interesting ride. During the journey, I’ll highlight the warning signs and provide an explanation into each. Let’s start.

    Day 1

    Out of the blue, I was contacted via email by someone calling themselves “Andrew Walter” - purportedly an employee at Bank of America. The first immediate sign that something is not quite all it seems here is that the email address used is in fact from the contact form on this site. What’s significant about that ? Well, there are a variety of techniques used by cyber criminals to gain access to legitimate email addresses. One known and widely used technique is the scraping of email addresses from websites and social media - in fact, the most notable is LinkedIn.

    Despite its age and somewhat basic approach, it still works very well. Why didn’t I secure it ? Simple. The contact form on this site also doubles as a honeypot. You’d be surprised what lands in here - as this “campaign” did. For the record, Phenomlab does not retain any information from this contact form. The initial text in the email might have been relatively convincing if it hadn’t contained a ”glow in the dark” grammatical error within the first line. What I’m alluding to here is that the email may as well have arrived complete with sirens and flashing lights. Here’s a snapshot

    Dear Mark Cutting. “I added you to my professional network in order to share a confidential proposal with you please contact me on my private email: andrewwalter411@gmail.com for briefing on proposal since i can not send attachment via linkedin”.

    Actually, you didn’t. I received no such request. Let’s have a look at the initial baiting technique. Who writes an email using the full name of a person without addressing them in the business (or even personal for that fact) sense ? In addition, why would you wrap what you want to say in quotes ? Finally, “I can not send attachment via LinkedIn” - actually, I received two from trusted sources in the same platform a day earlier. This email was so cringe worthy, I thought it rude to not reply ?

    Andrew, Can I ask what this is in relation to please ? Thanks

    That’s the hook that a scammer needs. After this, the response is a lot more detailed as the criminal plays out the story. I’m going to highlight the areas of interest here as I go, and have attached the full text in order to keep this article sane.

    I will start by saying thanks for your response…How is your family doing? I hope okay.

    Good start. Make it look like you know me personally and commence with the pleasantries - even though you in fact know nothing about me, and, in reality, couldn’t care less.

    My proposal is very important to me so please I want you to take the content of this mail very serious. All I want is an honest business transaction between us.

    This is anything but honest

    Day 2

    First of all, I will start by introducing myself. My name is Andrew Walter, I am currently working with Bank of America. I have been working here for 17 years now, and I have a good working record with my bank.

    That’s strange. According to the array of fake Andrew Walter (Bank of America) LinkedIn profiles, you’ve been there for 12 years. Did you step into a time machine and not tell anyone ? Perhaps you banged your head and lost 5 years in the process. What’s more than likely is that like most bad liars, you’ve lost track of what you told one person as oppose to the next. At least you tried to enforce a bit of trust with your statement around “I have a good working record with my bank”.
    1614967980-136791-linkedinpng.webp1614967988-257399-linkedin2png.webp

    I am also the personal accountant to Engineer (Lex Cutting ), a foreign contractor who has an investment account with my bank with a huge sum of money in it.

    Note the misplaced bracket here, and also note, that there is no “Lex Cutting” in my family tree. Am I a grammar snob ? No, but I expect a “business transaction” (if you can call it that) to at least not contain basic grammatical errors.

    My late client was a chemical consultant contractor with Royal Dutch Shell until his death in a fatal car accident while at France on sabbatical with his entire family. The accident unfortunately took the lives of the family members comprising of himself, his wife and two kids in the summer of 2007 may their soul rest in perfect peace.

    He banked with us here at Bank of America and had a very huge sum of money in his account which has still yet not been claimed by anybody as there was no living will in place when he died.

    “May their soul rest in perfect peace” and “A very huge sum of money” - instant alarm bells owing to the poor grammar. If you’re working under the pretence of being an educated individual employed by a tier 1 bank, you’re not doing a very good job.

    The amount of money involved here is about $15,812,664 (Fifteen Million, Eight Hundred and Twelve Thousand six hundred and sixty four US Dollars.) in account with indefinite interest.

    Holy s***, I’ve won the lottery !! Contain yourself man, and remember, its a fake ! Ok, composure resumed.

    Since the death of my client; my bank and I have made several inquiries to his embassy to locate any of his extended family members or relatives but this has proven unsuccessful. I came to know about you in my search for a person who shares the same last name as my late client.

    Yes - I and thousands of others no doubt. How lucky I’ve been selected for this “unique opportunity”.

    employed the services of LinkedIn search solely for this purpose as I feel it would not have been the last wishes of my late client for his whole life work to be transferred to a government (Es cheat) he had always complained of their unfavorable public monetary policies, taxes and so on while he was alive.

    Ok, so let me get this straight. You’ve trawled LinkedIn looking for “beneficiaries” when there are other far more orthodox and reliable channels to obtain this information. I can smell the sweat and toil of poorly conducted fraud here. Oh, and by the way, “Es cheat” is actually one word (ESCHEAT).

    My bank has issued me several notices to provide the next of kin or the account risk been es cheat within the next 10 official working days. The last notice for claim came to my desk last week. I am contacting you to assist me in repatriating the funds left behind before they are declared un-serviced by my bank. I am seeking your consent to present you as the next of kin of my late client since you share and bear the same last name.

    As such, the proceeds of the account can be paid to you as soon as you contact my bank and apply for the funds to be released to you as the next of kin. If we can be of one accord, I see no reason why we would not succeed. We both have to act swiftly on this matter in other to beat the deadline es cheat date.Please get back to me immediately for us to proceed.

    Wait a minute. If I’m the sole beneficiary, why do you want half ? Sounds like easy money to me. And the usage of “one accord” is somewhat “odd”.

    I am after the success of this transaction with your full co-operation. All I require is your honesty and full co-operation to enable us see this cool deal go through.

    I bet you are. “Cool deal” ? I thought I was taking to a professional here, not a school kid. Seems like our man has let his guard down for a split second and now his “Inna Gangsta” is shining through.

    I guarantee you that this will be executed under a legitimate arrangement that will protect you and me from breaching USA laws. I want to also inform you that I am a very religious person and I cannot tell a lie because of my strong believes; I would expect the same from you.

    Oh please, do me a favour. Pull the other one - its got bells on.

    I will attach a copy of my international passport in my next mail for authenticity so we have equal ground to trust each other. If you are interested in my proposal I will send you more information directing you on further procedure on how we can claim the money in the account successfully. If this proposal is alright by you then kindly get back to me.

    “Alright by you” - there’s that superb [sic] usage of business language again. This guy is awesome.

    The content of this mail should be treated with utmost confidentiality and a quick response from you will be highly appreciated. However, if you are not interested in this proposal, please accept my apologies for sending you the message and kindly delete message, I promised that you will never hear from me. I anticipate your co-operation.

    Of course. You wouldn’t want local law enforcement or the ”feds” knocking at your door now, would you ?

    Day 3

    This by now is so hilarious that I just had to respond.

    Hi. This sounds great. What would the next steps be ? Eagerly awaiting your response.

    And, without delay, here’s the response

    Dear Mark Cutting. I thank you for responding to my mail, I want to stress again that this transaction is very legitimate and there is no risk involved as I am the personal accountant to Late Engineer (Lex Cutting ) anything I say concerning this will be followed by the bank Executives.

    I bet. Actually, I’m struggling to follow your appallingly bad grammar here, but I expect you have your “very legitimate” reasons.

    However, before we can proceed further, I want you to assure me that you will be honest during the transaction and as soon as the funds is transferred to you we can meet in person and share money peacefully. You should understand that this transaction can be successful if we work together and as soon as I give you all legal procedure you will receive the funds from my bank, so I really need your assurance before we shall proceed.

    Wait - you want me to be honest ? Who’s scamming who here ?? What a complete scumbag.

    As I read your email I am very convinced with you and serious about this arrangement process as such, I would want you to take this serious too. My personal instinct directed me to contact you and I hope it was not a wrong thing to do.I shall direct you on the process of the claim; we shall start by sending a formal application to this effect. I will send you the text for the claims and transfer application to this effect. Thereafter, the bank will request of you the relevant back up documents to your claim and application according to the demand of our probate law for transfer of funds.

    Once you have provided the Bank with their demands, they would now be under legal obligation to transfer the funds to bank account provided by you. As part of the procedure of the claims, the documents that will be required from you will have to be acquired through legal procedures as the application of claim will be complimented with a legal award we shall have to seek from our law Court here. Be assured that the procedures to be adopted in effecting the transfer in your favor will be official and legal which will protect us from any breach of the law, We have the next 10 official working days.

    Right. Sounds fairly “straightforward”.

    Note: High confidentiality is required at all times. Do not tell anyone about this because, it might be unsafe for both of us. It would be safer for us to communicate by email for now as we have the trust. I hope you see reason with my decision on us talking by mail for now. As soon as the money has been transferred to your account, I will look for a country of our choice where we can see in person and subsequently share the funds in the ratio as discussed earlier.

    I can assure you it won’t be unsafe for me, but it probably is for you -“…now we have the trust”. Note, that the scammer gains confidence here, and starts making some fairly basic mistakes.

    Above all, I personally count on God to facilitate our plan and understanding, to produce not just success but also peaceful sharing of the funds at the end of the day and a wealthy family business relationship between us. I also pray for establishment of cordial relationship between us, God being our helper.

    I agree - you’re definitely going to need all the help you can get here. You’re not getting anything from me, so divine intervention is probably the only thing you have left.

    As soon as I hear from you and receive your assurance, I will send you the Text of Application for you to contact my bank for the release of the funds in the account of (Lex Nicholl) to your account as his next of kin.

    Hold it right there ! Who is Lex** “Nicholl”** ? Major alarm bells here. Looks like this guy has his wires crossed or didn’t get good morning injection of caffeine. This is a glaring oversight and I’m guessing all those lovingly created campaigns have a similar fault.

    would advice that you follow all the steps and procedures which I will give you so that we can get to the end of this transaction quickly. I need you to send a copy of your international passport to me and I will send mine as soon as I receive your reply indicating understanding from both of us.

    Of course. You need my passport. How undeniably stupid of me to think that you could complete this “transaction” without stealing the holy grail of personally identifiable documents in the process and using it like the gift that keeps giving for your other criminal campaigns (I sincerely hope they are better than this one).

    Day 4

    Time to turn up the heat a bit

    Hi. Can you send me the claims transfer forms for review ? Thanks

    This guy is like a dog on heat and he’s well and truly bitten this

    Dear Mark Cutting. I hope you and your family are well am so sorry for my late response as i read your email I was convinced, and I want you to understand that I need proper confirmation as I states below to be more in assurance of doing this transaction with you. The documents that will be required from us will have to be acquired through legal procedures as I explained, the application of claim will be complimented with a legal back up confirming this as a legitimate transaction, I have the account details with all access codes and will give it to you once it is required by the bank, also with me here all approvals will be provided and the transfer released to you.

    We are going to keep our communication on email for now to ensure that we are under absolute security due to high level call interception here in United States I would like you to see it with me that security is very necessary we have to be on email or text messages until the transaction is completed and I will visit you to implement our sharing.

    Yes, I agree that security is “very necessary” and also appreciate you do not want roughing up by “the feds” anytime soon. Let’s keep the communication on email so I don’t start to question who you are ? A quick side note here - if you want a secure channel, email is completely the opposite unless its been encrypted - which this hasn’t, and could be subject to eavesdropping. And, as a way of putting my mind at rest, here’s a lovely fake passport for your viewing pleasure. To the untrained eye, this could look convincing, but it a fake. One of the key identifiers here are the “wavy lines” over the picture. This is in fact a security watermark, and is unique for each passport issued. The lines will never repeat each other - if you look carefully at the below, the lines do in fact repeat.
    1614968131-783791-passport-fakepng.webp
    Below is an actual fake passport that was used in a scam a number of years ago. You’ll notice that this one is slightly less complex as it has the watermark missing, but is still fake, nonetheless.
    1614968502-542440-fake-passport-examplejpg.webp

    The transfer in your favor will be official and legal which will protect us from any breach of the law. Whatever the cost of his transaction will be, is going to be on both of us which I believe that you will not let me handle all the process alone.

    Of course not. You wouldn’t want to have to share any of the spoils, would you ? And just like any other “business transaction” you don’t want to be spending any of your money unnecessarily. Interesting that he’s actually used the US English “favor” rather than the UK English of “favour”. Pity he’s not been so diligent elsewhere. I know…let’s try and spend mine.

    I will give you the text application letter of the transfer request for our ledger department and also details on the way forward with the transaction once you have agreed with the following

    Are you ready to maintain the high level of confidentiality required for the successful conclusion of this transaction?

    Are you promising me that your account can be able to carry a transaction of such magnitude without any problem

    Are you willing to accept 50% for your participation without any problems in collecting my share from you?

    “Yes, yes, yes !” Let’s do this thing, and I’ll also throw in a portable radio to make the deal even more “appealing”.

    I will need your help in directing me and investing part of my share in your country the investment will be under your control until I am able to take over or it can be a joint venture depending on your decision. as soon as i receive a copy of your passport or id document and i as well have attached a copy of my passport for you to see whom you are working with.

    Please reply as soon as possible if you in understanding with me so that we can proceed with the bank with text application.

    Day 5

    Now this is getting interesting. What this really means is that once I have your bank details, I won’t be making a deposit - only a withdrawal (from my account, of course). Time to contact the Bank of America - this guy is an absolute riot (anagram of idiot) and yes, I can’t spell either, or count.

    Dear Sirs, I write with reference to what I believe to be a 419 Nigerian scam, sent to my email address. I am a security expert by trade, and wish to report this to yourselves. I believe the “sender” is impersonating one of your employees. I have also enclosed a scanned PDF file of the “passport”, which I also believe to be fake. I’m currently entertaining this individual as a way of reeling him in so I can report him to the necessary authorities.

    Clearly, I have no intention of supplying any sensitive information, including my passport. Whilst I expect that you receive many emails of this nature, I would like confirmation that the enclosed photo in the passport is not in fact a Bank of America employee

    Sadly, absolutely no response from Bank of America. I expect that they receive thousands of emails like this on a regular basis. Oh well, onward and upward. Let’s not keep our friend waiting.

    Hi Mark , Thank you for your email, and understanding, we do not have much time to complete this transaction to avoid reaching the es-cheat date.i will start the preparation of the application text which will be submitted to the bank as official application to cover the estate by the family member of Late Mr. Lex Cutting.

    I will send it to you for review by tomorrow. As a side note, there’s that misplaced capital letter

    Well now, that’s more like it ! Now we’re best friends forever, we can lower our guard a bit and revert to informal language (well, formal in the sense that our author is suffering from capital letter displacement). Perhaps we caused a bit of suspicion in our last messages and want to be a bit more convincing ? I’m game if you are buddy. Let’s make this a bit more interesting.

    Hi Andrew, Thanks for the email. I’ve just moved house, and things are in a bit of a mess, so I cannot place my passport for a few days until I’ve finished unpacking - hopefully, this doesn’t cause you any problems. I can answer “yes” to all the questions below.

    In the meantime, to speed up the process, is there any way we can proceed whilst I attempt to find my passport ?

    Thanks

    Well, look at me ! I know exactly where my passport is and I haven’t moved house - we need a bit of time here to do some further digging, so I’m throwing him off the scent for a few days whilst I perform some background investigation and analysis. I let this go on for 6 days before responding - note, that previously, “Andrew” had warned me we only had 10 days to nail this “cool deal”.

    We’ve since passed that landmark, but interestingly, he’s not that worried it seems. Admittedly, at this point I thought of sending a copy of Jason Bourne’s passport which are readily available for download via a quick Google - http://www.indyprops.com/pp-bournepass.htm. However, despite my assumption that this person I’m dealing with is stupid, I don’t think there’s many people on this planet who haven’t heard of Jason Bourne or seen at least one movie from the franchise.

    Based on this simple conclusion, its not a wise move in my view as it means ending the story here (unless this guy has been living under a rock)…. and there’s so much more to tell yet ! Therefore, we’ll need to take another route. Let’s increase the stakes. Note that by this point, we’re up to day 5, and we only had 10 days to complete this “cool deal”.

    Its now day 11 after I’ve kept him waiting for 6 days intentionally.

    Day 11

    Hi Andrew, Sorry for the delay. I finally found my passport, and have scanned a copy. However, I’ve read that email isn’t secure, so I can either FedEx a copy to you (I’ll need an address of course), or I can provide a secure link for you to download a password protected zip file. I’ll email you the password for that under separate cover. Would this be ok ? Keen to get things moving. Thanks

    I can almost hear the cogs in motion as my best friend formulates a response. A spanner in the works and probably not on his “canned response” sheet. This guy now needs to up his game to stay in the running.

    Hi mark. I hope you and your family are well? thank you so much for your mail please scan and send the copy of your international passport to this email (andrewwalter166@gmail.com) will can communicate much better even while i’m right in my work place i can reply over there anytime. as soon i receive your reply we will be proceeding with the text of application.

    I will be waiting to hear from you.

    Yes, I bet you will. This is the response I expected (note the “new” email address highlighted in yellow above - why change this now ? Keep reading) - if I then dropped out afterwards, this guy would still have a copy of my genuine passport, and could (and undoubtedly would) use this to commit other types of fraudulent activity.

    Essentially, its all about the money, so if the primary campaign fails, there is a good chance the second one will succeed, which is why the passport is requested so early to avoid over investment in terms of time.

    Hi Andrew, I really don’t want to send my passport by email. Can you give me an address of where it can be sent (postal) or let me know if you’d be ok downloading the copy needed from a link I will provide ? Thanks

    “Hang him on a hook and let me play with him”
    1614968549-147228-hhoahjpg.webp
    I’m so bad. Let’s see how much he wants this. Pushing for the postal address risks blowing the (supposedly carefully planned) cover and exposing him. He can’t exactly give me an address in Africa now, can he ? I’ve already preempted this and laid the foundations for a honeypot trap. I need to explain myself a bit here for those reading this and scratching their heads with images of Winnie the Pooh and a honey jar, so bear with me.

    A honeypot is a computer system or landing page that is set up to act as a decoy to lure fraudsters and cyber criminals - its essential function is to detect, deflect or study attempts to gain unauthorized access to information systems that are not for public use. At the heart of this honeypot is a system that is capable of obtaining a wealth of information about the accessing user in terms of IP address, geographic location, and a whole variety of data that would allow the recipient to piece together a trail of breadcrumbs. Any seasoned cyber criminal knows about the existence of such technology (its not exactly new) and would typically use a TOR browser to connect to any links provided by the victim in order to avoid detection.

    The TOR network is a complex array of secured computer systems acting as “nodes” that traverse the internet using a variety of encryption mechanisms and connection masking, allowing the user to hide behind a number of random proxies that make it look as though he or she is accessing from a completely different geographical location. The TOR network was originally intended for use by the US navy, but found its way out and became the favourite watering hole for many a cyber criminal - and today, known as either the deep web, or worse, the dark web. Ok, that’s enough history and boring technical terms. Let’s get back on track. Essentially, I’ve created a hidden honeypot on this site and the only two people who have this link are myself, and our scammer friend. The page cannot be indexed or crawled by Google either. Time to up the stakes

    Hi Andrew, Any update to this please ? Thanks

    Day 12

    No response. Perhaps I’ve pushed this a little too far. Let’s see

    Hi Andrew, I’m concerned that I haven’t heard from you and don’t want to miss out on this amazing opportunity. Can you let me know what we need to do next please ? Thanks

    I honestly thought that he wouldn’t reply, but he did.

    Dear Mark Cutting. Hope all is fine with you and the family? i am writing to know if you are still interested with this transaction i need a copy of your international passport in other to know whom i’m working with for more verification as soon you send it down here

    Now, when I went to school, the UK was across from America and not down - hence the term “across the pond”. Did I miss something here ? A figure of speech perhaps, but more likely a slip of the tongue. Looking at a map “down here” would indicate south, surely ?

    we will be proceeding with the text of application to contact my bank for funds relic please update me as soon as possible.

    And here we have another schoolboy error. This guy thinks he can relax now he’s done his chore. Not only is the text clearly copied and pasted (with the formatting intact so he first line doesn’t match the rest in terms of font size), but much worse is the fact that he’s now using a different email address altogether and hasn’t even made any attempts to hide this. Clearly, he’s got a lot going on, and there are undoubtedly hundreds of “Andrew Walter” doppelgangers lurking in the shadows like something out of Michael Jackson’s “Thriller”.

    To understand this complete failure, let’s take a closer look - perhaps he’s got some sort of “Salesforce(esque)” campaign on the go where willing participants are directed to another email address for easy reference (milking)! The email address we started with was “andrewwalter411@gmail.com” which in itself isn’t very convincing. Now we’ve suddenly switched to “andrewwalter166@gmail.com” and also lazy again with our grammar as “Andrew Walter” is now “andrew walter”.
    1614968634-496024-emails.webp
    I suppose I could send him a Starbucks voucher so he can get a strong coffee and wake up, but, this is his gig, so I’ll let him play his hand.

    Hi Andrew, As I previously mentioned, I won’t send my passport via email because I was told it wasn’t secure. Instead, I’ll provide a link to a secure website where it can be downloaded as a zip file. I’ll also provide the password for the zip file so you can extract it. I’ll get this over to you today. Thanks

    Now we’re “upping the ante” a bit. Not only do we respond to the original email, but also to the new one with the same message above. I’m relatively sure at this point that our friend isn’t exactly an experienced fraudster, and probably won’t even notice his own mistake. Wait for a bit……. then send the link. Note that the link itself has been redacted for obvious reasons and is not the original.

    Dear Andrew, I’ve scanned a copy of my passport to PDF and placed it in a password protected ZIP file. It can be downloaded using the link below. https://[redacted]/KCXXu4MN8G6FZqFt4Mb7hQfRZXmHA3Fn/securedownload/ Let me know as soon as you’ve downloaded the ZIP file, and I’ll send you the password in a separate email. Thanks

    I wasn’t really expecting this guy to bite if I’m being honest, but never say die - he’s just fallen straight into the honeypot (or should I say, “boiling pot”)

    Hi, the link is infected i can not open it my system refused to run the link. send it via pdf which i can view before download or jpg.

    Actually “Andrew”, the link **isn’t **infected. I understand your frustration though, as its very annoying having your time wasted by a moronic idiot who seems to lack the ability to string even basic sentences together…… Alright - that’s enough of that. The thing is “Andrew”, you didn’t follow my instructions. Not that this really matters at the moment anyway as I have got what I came for. The string on the carrot has just been made shorter. I know at this point, you can almost taste it, but I’m not finished with you just yet.

    Hi Andrew, The link works fine on my PC. its a password protected ZIP file created by 7Zip. If you use this to extract, you’ll need to enter the password to extract the PDF which I’ll send you under separate cover. Regds

    He’s in for a bit of a surprise when he gets around to opening that Zip file. There’s a PDF there all right, but it’s certainly not my passport. In fact, “Andrew” has had three attempts at downloading that file
    1614968689-559410-file-downloadpng.webp
    According to the honeypot, it would appear that he’s operating out of Randburg (Johannesburg, South Africa) - a very well known fraud hotspot.
    1614968849-529265-locationpng.webp
    The GEO information is provided courtesy of https://db-ip.com/41.113.125.214

    If those coordinates are accurate, then the local law enforcement aren’t too far away. Have a look below
    1614968903-340115-policepng.webp
    In fact, about 12 miles away (dependant on exact location of course, which the local ISP can provide when requested by law enforcement agencies)
    1614969237-763652-directions1.webp

    Day 13

    The next steps here are quite obvious. Pass it onto the local authorities to investigate, with a copy of all material received thus far

    Dear Sirs, I write with reference to an incident where a scammer in your location has trawled LinkedIn and obtained my address with a view to commit coercion and fraudulent activity. The IP address that this fraud attempt has originated from is https://db-ip.com/41.113.125.214. I have a complete record of all activity, plus a copy of what I believe to be a stolen passport.

    I am a security professional by trade, and wish to report this as criminal activity. I have a complete evidence chain of emails relating to this particular event - the incumbent has requested a copy of my passport (which for obvious reasons I will not be providing), and no doubt will also attempt to acquire my bank details. This person is posing as “Andrew Walter” from Bank of America - there are several fake profiles on LinkedIn relating to this individual. I am also aware that local law enforcement can request the physically connected location for this address - you should find its about 12 miles away from your location.

    I have obtained this fraudster’s IP address via a honeypot on my website, which I purposely setup to extract this information. I would appreciate your cooperation in this individual’s apprehension, as it would appear that the same person is responsible for a number of similar campaigns designed to extract funds from others. I am based in the UK, but can be free to discuss as you deem fit. I have enclosed copies of all emails received so far, plus an example of the LinkedIn profiles which I believe are fake. Mark Cutting

    And the below read receipt shows that this email has been read (well, opened, at least)

    Your message was read on 16 May 2018 10:32:48 AM UTC. Final-recipient: RFC822; T0023694@saps.gov.za Disposition: automatic-action/MDN-sent-automatically; displayed X-MSExch-Correlation-Key: c1tMJuEijE6r4WJRtMhQlw== X-Display-Name: GPS:Randburg SC Admin

    its at this point where things become much clearer. This guy really hasn’t done his homework. He’s been conversing with me outside of US time zones (well, Johannesburg is only currently 1 hour ahead of the UK after all) which can only mean he either has severe insomnia, or isn’t actually based in the USA. I wonder which one it could be ? Perhaps he should see a doctor and get some pills for that…. 🙂 I’ve since sent “Andrew” another email, but unfortunately, he hasn’t replied. I guess he’s “busy” with his next victim.

    Hi Andrew,

    I’m a bit concerned I’ve not heard from you, and with the deadline approaching, I really do not want to miss out. Can you let me know if you were able to open the zip file with 7zip as I previously mentioned ?

    When you try to extract it, you’ll need a password which I’ll provide to you once you confirm you’re able to open.

    Please keep me updated.

    Thanks

    The ironic thing here is that “Andrew” in fact already has the password for that zip file I sent him ! If he’s the hotshot he makes out to be, then I’m sure he’ll work it out. In the meantime, I’m guessing you all want to know what that zip file contained ? Well, I did say it was a PDF, but its not my passport. Here you go.
    1614969001-333126-pdf1png.webp

    Conclusion

    Sadly, there’s been no response to the email I sent to SAPS (South African Police Services). Oh well. They have all the evidence they need, although in fact, no actual “fraud” has been committed. That effectively means that “scoping out” a potential victim and attempting to reel them in isn’t actually an offence. Although identity impersonation certainly is and I’d be surprised if they were not interested in this.

    So there you have it - a walk-through of what to look for in these types of scam. Here’s the highlights

    No official institution like the Bank of America is going to allow its employees to conduct business over a GMAIL account. In all honesty, faking the bankofamerica.com domain would have been much more convincing, and wouldn’t have taken much effort either. After a quick iteration of the real name, I found the below If an email supposedly comes from the US, then why are all emails being sent outside of their working hours ? Any transaction of this sort would never be conducted over email anyway - for this amount (if this were indeed real), it would have to be completed face-to-face in the presence of bank officials, lawyers, compliance, and a whole raft of others. No institution is going to request a copy of any identifiable details (passport, bank accounts, etc.) over email. Poor grammar is an immediate warning sign. You need at least a decent grade in English if you are going to pretend to be someone you’re not Bad spelling is another. There are so many errors here and it makes any campaign stand up and shout “hey, I’m fake !”

    Hope you enjoyed this somewhat absurd journey.

    Keep safe out there, folks.

  • phenomlabundefined

    Keep It Simple, Stupid...

    Blog
    1
    0 Votes
    1 Posts
    298 Views
    phenomlabundefined

    tech.jpeg
    Ever heard of KISS ? Nope - not these guys

    kiss.jpeg
    What I’m referring to is the acronym was reportedly coined by Kelly Johnson, lead engineer at the Lockheed Skunk Works (creators of the Lockheed U-2 and SR-71 Blackbird spy planes, among many others), which formed the basis of the relationship between the way things break, and the sophistication available to repair them. You might be puzzled at why I’d write about something like this, but it’s a situation I see constantly – one I like to refer to as “over thinker syndrome”. What do I mean by this ? Here’s the theory. Some people are very analytical when it comes to problem solving. Couple that with technical knowledge and you could land up with a situation where something relatively simple gets blown out of all proportion because the scenario played out in the mind is often much further from reality than you’d expect. And the technical reasoning is usually always to blame.

    Some years ago in a previous career, a colleague noticed that the Exchange Server (2003 wouldn’t you know) would suddenly reboot half way through a backup job. Rightly so, he wanted to investigate and asked me if this would be ok. Anyone with an ounce of experience knows that functional backups are critical in the event of a disaster – none more so than I – obviously, I gave the go ahead. One bright spark in my team suggested a reboot of the server, which immediately prompted the response

    “……it’s rebooting itself every day, so how will that help ?”

    There’s always one, isn’t there ? The final (and honestly more realistic suggestion) was to enable verbose logging in Exchange. This is actually a good idea, but only if you suspect that the information store could be the issue. Given the evidence, I wasn’t convinced. If there was corruption in the store, or on any of the disks, this would show itself randomly through the day and wouldn’t wait until 2am in the morning. Not wanting to come across as condescending, I agreed, but at the same time, set a deadline to escalation. I wasn’t overly concerned about the backups as these were being completed manually each day whilst the investigations were taking place. Neither was I concerned at what could be seen at this point as wasting someone’s time when you think you may have the answer to what now seemed to be an impossible problem. This is where experience will eclipse any formal qualifications hands down. Those with university degrees may scoff at this, but those with substantially analytical thinking patterns seem to avoid logic like the plague and go off on a wild tangent looking for a dramatically technical explanation and solution to a problem when it’s much simpler than you’d expect.

    After witnessing the pained expression on the face of my now exasperated and exhausted tech, I said “let’s get a coffee”. In agreement, he followed me to the kitchen and then asked me what I thought the problem could be. I said that if he wanted my advice, it would be to step back and look at this problem from a logical angle rather than technical. The confused look I received was priceless – the guy must have really though I’d lost the plot. After what seemed like an eternity (although in reality only a few seconds) he asked me what I meant by this. “Come with me”, I said. Finishing his coffee, he diligently followed me to the server room. Once inside, I asked him to show me the Exchange Server. Puzzled, he correctly pointed out the exact machine. I then asked him to trace the power cables and tell me where they went.

    As with most server rooms, locating and identifying cables can be a bit of a challenge after equipment has been added and removed, so this took a little longer than we expected. Eventually, the tech traced the cables back to

    ………an old looking UPS that had a red light illuminated at the front like it had been a prop in a Terminator film.

    Suddenly, the real cause of this issue dawned on the tech like a morning sunrise over the Serengeti. The UPS that the Exchange Server was unexpectedly connected to had a faulty battery. The UPS was conducting a self test at 2am each morning, and because the bypass test failed owing to the burnt battery, the connected server lost power and started back up after the offending equipment left bypass mode and went online.

    Where is this going you might ask ? Here’s the moral of this particular story

    Just because a problem involves technology, it doesn’t mean that the answer has to be a complex technical one Logic and common sense has a part to play in all of our lives. Sometimes, it makes more sense just to step back, take a breath, and see something for what it really is before deciding to commit It’s easy to allow technical expertise to cloud your judgement – don’t fall into the trap of using a sledgehammer to break an egg You cannot buy experience – it’s earned, gained, and leaves an indelible mark
  • phenomlabundefined

    New blog section added

    Announcements
    1
    0 Votes
    1 Posts
    157 Views
    phenomlabundefined

    I’m excited to announce that a new blog section has been added 😛 The blog is actually using Ghost and not NodeBB, and also sits on it’s own subdomain of https://content.sudonix.com (if you ever fancy hitting it directly).

    We’ve moved all the blog articles out of the existing category here, and migrated them to the Ghost platform. However, you can still comment on these articles just like they were part of the root system. If you pick a blog article whilst logged in

    7e61c35b-2304-4c06-bda2-34da52252e1a-image.png

    Then choose the blog article you want to read

    7ca5089e-cf7e-4050-b951-5426fd1c6ec3-image.png

    Once opened, you’ll see a short synopsis of the article

    1bc086b4-5968-4e81-bc47-70de263b2275-image.png

    Click the link to read the rest of the post. Scroll down to the bottom, and you’ll see a space where you can provide your comments ! Take the time to read the articles, and provide your own feedback - I’d love to hear it.

    3f712e7c-475d-42d4-a5ca-b4becff6cc2e-image.png

    The blog component is not quite finished yet - it needs some polish, and there’s a few bugs scattered here and there, but these will only manifest themselves if a certain sequence of events is met.