Strategic Source Evaluation - Credibility, Validity, Relevance, and Types of Evidence
The core of your analysis is based on your evidence.
Watch Promo
The course focuses on determining the types of data collected as evidence, the types of evidence, how to evaluate the evidence for validity, the sources for credibility and the results for relevance to leadership and stakeholder needs, Students learn how to verify sources while assessing evidence for usability. The course provides step-by-step guidelines for evidence evaluation. In a crisis situation, social networks are overloaded with situational updates, calls for relief, reports of new developments, and rescue information. Reporting the right information is often critical in shaping responses from your organization during critical situations.
Your Instructor
Former adjunct professor of Cyber Intelligence, Counterintelligence, and Cybercrime (Utica College) and Information Security Risk Management (Clark University). Experienced in cyber intelligence lifecycle services and support, cyber counterintelligence services and analysis, active defense and cyber operations. Commercially teach Cyber Intelligence (Anonymity, Sockpuppets, Cyber Collection, Clandestine Cyber HUMINT, Socio-Cultural Aspects of Intelligence, Lifecycle, Critical Thinking, Cognitive Bias, Methods and Types of Analysis and Methods, Structured Analytic Techniques, Analytic Writing, BLUF/AIMS Delivery, and Dissemination), Jihadist Online Recruitment Methods, cyber influence operations, high-value target development, deception planning, deception operations management, Middle Eastern Cyber Warfare Doctrine, adversary dossier development and social-cultural analysis, jihadist training and gaming as a method of training, information and intelligence sharing, threat intelligence platform selection, non-inclusively.
Jeff Bardin is the Chief Intelligence Officer for Treadstone 71 with clients on 4 continents. In 2007, Jeff received the RSA Conference award for Excellence in the Field of Security Practices. His team also won the 2007 SC Magazine Award – Best Security Team. Jeff sits or has sat on the Board of Boston Infragard, Content Raven, Journal of Law and Cyber Warfare, and Wisegate and was a founding member of the Cloud Security Alliance. Jeff served in the USAF as a cryptologic linguist and in the US Army / US Army National Guard as an armor officer, armored scout platoon leader.
Mr. Bardin has extensive experience in cyber intelligence lifecycle services, program builds, targeted research and support, cyber counterintelligence services and analysis, deception planning, and cyber operations. He teaches Cyber Intelligence and Counterintelligence (Anonymity, Cyber Personas, Collection management, Clandestine Cyber HUMINT, Socio-Cultural Aspects of Intelligence, Critical Thinking, Cognitive Bias, Methods and Types of Analysis, Mitre ATT&CK, Structured Analytic Techniques, Analytic Writing, Briefings, and Dissemination), open source intelligence, strategic intelligence, operational/tactical/technical intelligence, and methods in media manipulation identification.
He has BA in Special Studies - Middle East Studies & Language from Trinity College and an MS in Information Assurance from Norwich University. Jeff also attended the Middlebury College Language School for additional language training. Mr. Bardin also spent two+ years studying Russian history, literature, political systems, and language. He lived and worked in the Mediterranean area, Europe, Australia, Singapore, Malaysia, the Persian Gulf Region, and the Kingdom of Saudi Arabia. Jeff was an adjunct instructor of master’s programs in cyber intelligence, counterintelligence, cybercrime and cyberterrorism at Utica College. Mr. Bardin has also appeared on CNN, CBS News Live, FoxNews, BBCRadio, i24News, BBN, and several other news outlets and has contributed bylines to Business Insider non-inclusively.
We started teaching these courses in 2009 and have continued to update and hone them while maintaining true to the intelligence community standards. We have since built cyber threat intelligence programs for Fortune 500 firms and government organizations on four continents while providing targeted research against adversaries and nation-states.
Treadstone 71:
We founded the company in 2002 and started creating cyber personas and infiltrating al-Qaeda sites collecting information and sharing it with various US-based organizations in 2004. In 2009-10, we started teaching Cyber Intelligence, Cyber CounterIntelligence and Cyber Crime courses at the master’s level at Utica College where we established the intelligence program. After three years of teaching at the academic level, we switched to the commercial space honing the courses to CIA/DIA style tradecraft as aligned to the cyber environment using the skills acquired in 2004. Since that time, we have continued to update the courses using real-world case studies as part of the training.
We have kept the company purposely small and now offer the training courses (www.treadstone71.com/cyber-intelligence-trainingan... www.cyberinteltrainingcenter.com) as well as Cyber Threat Intelligence maturity assessments, strategic and program planning, active research, collection, and reporting. We also perform Threat Intel Platform assessments, selection, and rollout activities for clients. We have clients in the US, EU, Australia, and Asia with active proposals in the Middle East. My personal background is as an Arabic Linguist (USAF / NSA), Russian Linguist, and CISO financial services, government contracts, insurance, and cybersecurity vendors. We have also acted as a critical resource for government CISOs in the past authoring their agency strategic plans, program plans and responding to Congressional inquiries on their behalf.
Jeff has spoken at RSA, NATO CyCon (Estonia), the US Naval Academy, the Air Force Institute of Technology, the Johns Hopkins Research Labs, Hacker Halted, Malaysian Cyberjaya, Secureworld Expo, Hacktivity (Budapest), IS2 Prague, London (RSA), ISSA, Security Camp (Cairo), and several other conferences and organizations.
Mr. Bardin has authored books and contributed chapters to several other books most recently Current and Emerging Trends in Cyber Operations from George Washington University. Recently edited and provided content for Understanding Computers: Today and Tomorrow by Deborah Morley, Charles S Parker - 11th edition (March 2006 release). Reviewer for Building an Information Security Risk Management Program from the Ground Up (Evan Wheeler), Author Chapter 33 Computer Information Security Handbook 5th Edition - SAN Security. Author Chapter on Satellite Security - Computer Information Security Handbook 6th Edition. Author - The Illusion of Due Diligence - Notes from the CISO Underground (April 2010 release).
Treadstone 71 is a pure play intelligence company focusing on targeted research of adversaries building in-depth dossiers recording methods, tactics, techniques, procedures, known associates, memberships and psychological profiles. We author Current, Research/Foundational, Advisories, STEMPLES Plus, and Estimative Intelligence reports. We create profiles of high value targets including ‘know your customer’ profiles delivering assessments and gaps in protections with recommendations and opportunities.
We are known for building Strategic Intelligence Programs from vision, mission, guiding principles, goals, objectives, 36-month plans, policies, procedures, process flows, SOPs, KPIs, CSFs, training and awareness programs for intelligence. We also help establish internal intelligence community programs from technical and tactical to operational and strategic including physical, competitive, business, and cyber.
We have taught classes to and/or worked with/for:
AIB, American Express, Capital One, NATO, Belgian Military Intelligence, Commonwealth Bank, Bank of America, ING, NCSC NL, American Electric Power, Nationwide, Battelle, Standard Chartered, Columbus Collaboratory, Anomali, Defense Security Services, PNY, Dell Secureworks, HPE Security, EclecticIQ, Darkmatter (AE), General Electric, General Motors, PNC, Sony, Goldman Sachs, NASA, DoD, East West Bank, Naval Air Warfare Center, VISA, USBank, Wyndham Capital, Egyptian Government, DNB Norway, Euroclear, Malaysian Cyberjaya, People's United Bank, Baupost Group, Bank of North Carolina, Cardinal Health, Huntington, L Brands, OhioHealth, Fidelity Investments, Citi, Citigroup, T. Rowe Price, Wells Fargo, Davis Polk, Thrift Savings Plan, Discover, Equifax, Blackknight Financial Services, Schwab, GM, FRB, Intercontinental Exchange (ICE), Citizens Financial Group, Cleveland Clinic, Scottrade, MetLife, NY Life, Essent, Harvard University, Charles River Associates, Synchrony Financial, In-Q-Tel, TD Ameritrade, First Citizens Bank, M&T Bank, Western & Southern, American National Bank of TX, National Reconnaissance Office, OCBC Bank Singapore, Spentera, FBI, W.R. Berkley, F-Secure, People’s United Bank, Stellar Solutions, Lockheed Martin, Harvard Pilgrim, Symantec, State of Florida, Deloitte, Ernst and Young, Mitsubishi UFG Trust and Banking Corporation, Target, Tri Counties Bank, Mass Mutual, Tower Research, Latham and Watkins LLP, Geller & Company, KeyBank, Northern Trust, Fannie Mae, BB&T, Blue Cross Blue Shield Michigan, Farm Credit Services of America, Aviation ISAC, Regions Financial Corporation, Intercontinental Exchange (The ICE), Vista Equity Partners, JP Morgan Chase, Archer Daniels Midland, Nacha, Barclays, Options Clearing Corporation (OCC), Expo2020, Abu Dhabi Smart Solution's and Services Authority, Merck & Co., Inc Nomura International, ING, Finance CERT Norway, iPipeline, BBVA, PenFED, Santander, Bank of America, Equifax, BNY Mellon, UBS Group, OCC, Verizon, Vantiv, Raymond James, Bridgewater Associates, Bank of America Merrill Lynch, BBVA, Promontory Interfinancial Network, Bank of Canada, Credit Suisse, HSBC, Church of Jesus Christ of Latter Day Saints, Ocean First Bank, International Exchange, Splunk, Vero Skatt, Ernst & Young, Relativity, Ultimate Software, Vista Equity Partners, Aetna, QBE Insurance Group, ACI Universal Payments, Betaalvereniging Nederland, Dutch Police, Motorola Solutions, Intel Corporation, Salesforce, Singapore Ministry of Defence, Australia and New Zealand Banking Group Limited (ANZ), National Australia Bank Limited, non-inclusively (as well as several other firms by proxy as they hire qualified intelligence professionals trained by Treadstone 71).
Focus on targeted research of adversaries building in-depth dossiers recording methods, tactics, techniques, procedures, known associates, memberships and psychological profiles. Author Current, Research/Foundational, PESTELI, deception planning and operations, psychological operations, and Estimative Intelligence reports. Create profiles of high value targets including ‘know your customer’ profiles delivering assessments and gaps in protections with recommendations and opportunities.
Strategic Intelligence Program builds from vision, mission, guiding principles, goals, objectives, 36-month plans, policies, procedures, process flows, SOPs, KPIs, CSFs, training and awareness programs for intelligence. Building internal intelligence community programs from technical and tactical to operational and strategic including physical, competitive, business, and cyber.
Course Curriculum
Strategic Source Evaluation - Credibility, Validity, Relevance, and Types of Evidence
The core of your analysis is based on your evidence.
Watch PromoFrequently Asked Questions
How is this course different from the Certified Threat Intelligence Analyst – Cyber Intelligence Tradecraft Certification course?
We found a need to assist organizations to best understand the strategic functions of intelligence. Although there is some overlap in this course, the course goes into greater depth expanding well beyond traditional IT-type threat intelligence building the foundation for supporting decision-making outside of IT. There is some review for those who have taken previous Treadstone 71 courses but this course is the natural next steps in establishing a resilience, and sustainable cyber threat intelligence program. The course moves the functions and capabilities to a valid corporate asset.
Will the course offer the same types of hands-on exercises that make Treadstone 71 training the gold standard?
We deliver several hands-on exercises complete with templates and examples. Our intent is to send each student back to their corporate environments armed with the knowledge necessary to immediately enhance their existing programs or, start new programs with a foundation rooted in excellence.
STRATEGIC ANALYSIS |
ESTIMATIVE AND WARNING ANALYSIS |
Data, Information, Knowledge, and Intelligence |
The Role of Warning Intelligence |
Knowledge Generation |
Key Warning Factors in Preparations |
Explicitly versus Tacit Knowledge |
What Is Warning? |
Principles of Knowledge Management |
Intentions versus Capabilities |
Monitoring your Business Environment |
The function of Warning Intelligence |
Analysis Projects |
Indicators and Indications |
Analysis Cycle |
Strategic versus Tactical Warning |
Briefing |
What is a Warning? |
The Management Brief |
Warning as an Assessment of Probabilities |
Starting the Project |
Warning as a Judgment for the Stakeholder |
Project Brief Checklist |
Indicator Lists: Compiling Indications |
Collection Planning |
Fundamentals of Indications Analysis |
Attributes of Sources - Source-Centered Collection Plan |
Compiling Indications |
The Collection Plan |
Use of Indicator Lists |
Segmentation of Sources |
Extracting Indications Data |
Valuation of Sources |
The Nature of Cyber Indicators |
Separating Rumor from Fact |
Cyber Indications and Warnings |
Using Social Media like a Police Scanner |
The Nature of Cyber Indicators |
Monitoring and Verifying |
Importance of Cyber Indicators |
Image Verification |
Indications Chronology |
Video Verification |
Specifics of the Analytical Method |
Using the Crowd |
Presumption of Surprise |
Verification Process and Checklists |
Scope of Relevant Information |
Verification Tools |
Objectivity and Realism |
Intelligence Requirements |
Need to Reach Immediate Conclusions |
Prioritization |
Inference, Deduction and Induction |
Essential Elements of Information |
Acceptance of New Data |
Indicators |
Understanding How the Adversary Thinks |
Specific Information Requirements |
Consideration of Various Hypotheses |
Glossary and Taxonomy |
How Might they Go to Cyber War? |
Mission and Requirements Management |
Order of Cyber Battle Analysis in a Crisis Situation |
Tools to Use |
Cyber Order of Battle Methods |
Data to Collect |
Analysis of Cyber Mobilization |
Iterative and Continuous Feedback Loop |
Recognition of Cyber Buildup |
The Data Collection Plan |
Preparation for Cyber Warfare |
Executing the Plan |
Key Warning Factors in Preparations |
Collection from Friendly or Neutral Sources |
The preoccupation of Leadership / Stakeholders |
HUMINT |
Cyber Readiness |
Free-flow (Cooperation, rules, benefits, risks & issues, analysis) |
Exercises for Preparation versus Cyber Deployment |
Interviewing (Cooperation, rules, benefits, risks & issues, analysis) |
Magnitude and Redundancy of Preparations |
Sampling (Cooperation, rules, benefits, risks & issues) |
Cyber Wargaming |
Networking (Cooperation, rules, benefits, risks & issues) |
What is a Cyber Wargame |
Protecting your Sources |
Why run a Cyber Wargame |
Across Cultural Barriers |
Objectives |
Collecting from Unsuspecting Sources |
Success Factors |
Passive Collection |
Common flow |
Elicitation (Cooperation, rules, benefits, risks & issues) |
Common problems in setting up and running |
Elicitor - Qualities - Cyber Appearance |
STEMPLES Plus |
Collection from Public Domain |
Social, Technical, Economic, Military, Political, Legislative, Educational, Security |
Anatomy of OSINT |
Plus (Demographic, Religion, Psychological, catchall) |
Spelling, Singular/Plural, Acronyms, Jargon, History, Synonyms, Quasi-Synonyms |
Indicators of Change as Applied to STEMPLES Plus |
Applications of OSINT |
The ambiguity of STEMPLES Plus Indicators |
OSINT overload - Focus |
A Problem of Perception |
Collection from Images |
Considerations in STEMPLES Plus Warning |
Picture Analysis |
The Relative Weight of STEMPLES Plus Factors |
How to apply Intelligence from Image Collection |
Maintaining your STEMPLES Plus Indicators of Change |
When to do so |
Isolating the Critical Facts and Indications |
Imagery Intelligence output |
Guidelines for Assessing the Meaning of Evidence |
Collection from Things |
Hofstede Principles |
Back end collection and analysis |
Hofstede as Applied to STEMPLES Plus |
Where to apply the collection |
Adversary Baseball Cards |
When and How to apply the collection |
Country, Group, Campaign, Individuals |
Collection Outsourcing |
Reconstructing the Adversary's Decision-making Process |
Analysis |
Benching marking your adversary |
Introduction |
Adversary TTPs |
Attributes of strategic analysis |
Adversary Profiling |
Collector - Analyst Relationship |
Adversary Supply Chain |
Collector-Analyst Differences - Corporate alignment - All as one |
Skills and Education |
Strategic Analysis Cycle |
Tools and Their Application |
Anatomy of Analysis |
Principal Factors in Timing and Surprise |
Where, who, when, why, and how |
Examples of Assessing Timing |
Pitfalls |
Warning is Not a Forecast of Imminence |
Common pitfalls in analysis |
The Problem of Deception |
Bias |
Infrequency and Neglect of Deception |
Ethnocentric |
Principles, Techniques and Effectiveness of Deception |
Wishful Thinking |
Types of Deception |
Status quo |
Countering Deception |
Herding |
Judgments and Corporate Policy |
Previous Judgments |
Facts Don't “Speak For Themselves’’ |
Conventional wisdom |
What Do Top Stakeholders Need, and Want, to Know? |
Data and meta data |
Intelligence in Support of Policy? |
Data QA |
Assessing Probabilities |
Data processing and QC |
Improving Warning Assessments |
Data Credibility |
Factors Influencing Judgments and Reporting |
Source Validity |
General Warning Principles |
Data and Source Relevance |
Most Frequent Impediments to Warning |
Scoring Methods |
|
Data Preparation |
Appendix A – FORMS |
Managing incomplete data |
Key Assumptions |
Managing conflicting data |
Indicators / Observables Matrix |
Weighing Data |
Threat Situational Awareness |
Working with experts - |
Detection Indicators – Threat and Disposition |
Data Quantity versus Quality |
Threat Type – Description – Disposition |
Misperceiving Events |
Priority Intelligence Requirements – Collection Planning |
Premature closing |
Kill Chain Phase |
Confusing causality and correlation |
Types of Analysis |
Flawed analogies |
Decomposition |
Functions and Responsibilities |
Link Analysis |
Structured Analytic Techniques |
Pattern Analysis |
Link analysis/network charts |
Trend Analysis |
Timeline/Chronology |
Technical Baseline |
Network Analysis |
Functional Baseline |
Brainstorming |
Cultural Baseline |
Structured Brainstorming |
Tendency Analysis |
Virtual Brainstorming |
Cultural Analysis |
Nominal Group Technique |
Anomaly analysis |
Starbursting |
Semiotic Analysis |
Cross-Impact Matrix |
Anticipatory Analysis |
Morphological Analysis |
Volatility Analysis |
Quadrant Crunching |
Supply Chain Analysis |
Scenario Analysis |
Recomposition |
Mechanics of Scenario Analysis |
Synthesis |
When and why to plan |
Analyst - Stakeholder Interaction |
Success factors |
Uncertainty |
Design principles |
Decision-making strategies |
Attributes of a good scenario |
Challenges |
Flow of a Scenario Exercise |
Moving towards a Trusted Advisor Role |
Pitfalls in Scenario Analysis |
Cherry picking |
Alternative Futures Analysis |
Yes Manship |
Indicators |
Groupthink |
Indicators Validator |
Compliance Mandatory - Ethics as Identity |
Hypothesis Generation |
Legislation |
Formulation and testing |
Scope of compliance and ethics in analysis |
Theories, Forecasts |
Code of ethics for strategic analysis |
Testing |
Organizing a Strategic Analysis Function |
The Multiple Hypotheses Generator |
Getting started |
Diagnostic Reasoning |
Structure after strategy |
Analysis of Competing Hypotheses |
The right structure enables efficient/effective execution |
Argument Mapping |
Centralized versus Decentralized - a comparison |
Deception Detection |
Organizing a solid team |
Key Assumptions Check |
Design principles |
Outside In Thinking |
Functional and behavioral competency building |
Pre-Mortem Assessment |
Towards a world-class strategic analysis organization |
What If? Analysis |
Five Levels of Strategic Analysis Professionalism |
High Impact, Low Probability |
Profile of an analyst |
Devil’s Advocacy |
Functional competencies |
Force Field Analysis |
Behavioral competencies |
Maps |
Measuring competencies - Competency models |
Flow charts |
Job descriptions and hiring questions |
Frequency charts |
Accountability, Key Activities, Results |
Story boards |
|
Appendices |
This course teaches students how to think independently and stay away from the low-level tactical approaches we see in daily reports. Strategic, big-picture reviews and assessments that incorporate the social, technical, economic, military, political, legislative, educational, and security, plus demographics, religion, and the psychometric (STEMPLES Plus) aspects of an adversary are lost in today's world of current news posing as intelligence. Lecture, Hands-on, Apprenticeship, in-class exercises, student presentations, analytic products, templates, course material— 60 CPEs.