Assessing project safety without software knowledge
Level 2 - Value Investor
Welcome Avatar!
Many of you have asked for more information or a framework for assessing the security of a protocol, rather than just depending only on the external audits. These are good questions, especially as some projects launch or soft launch before an external audit has been completed.
We’ll start with a summary of the risks (why you should care) then cover some actions you can take to improve your odds of avoiding rugs, even if you can’t read code.
It isn’t just about hacks…
Some projects are badly designed then pumped to unrealistic and unsustainable valuations by charismatic leaders like Do Kwon (Terra / LUNA) and Daniele Sesta (Wonderland / TIME).
Other projects are just badly designed and doomed to fail – but failure can affect liquidity providers as well as direct investors (see Beanstalk Farms / BEAN).
But. Hacks are still a big risk.
Mirror Protocol lost $90 million this week. It seems they didn’t hire someone to manage their oracle.
A Conservative Rule
“The Principals should have a lot to lose in the venture”
This applies most strongly to early-stage ventures. By the time a project reaches LUNA, TIME, or HEX size the principals often have more to gain by rugging, in one form or another.
Lots to lose should certainly include the value of their unvested tokens (check the chain!) but it should go beyond. Examples of ‘costly signaling’ include:
Sweat equity invested in creating or perfecting a novel solution to a problem
Good reputation whether in crypto or another industry - mostly applies to doxed teams (but can apply to anons with a valuable reputation too)
Time commitment evidenced by e.g. leaving career to work full time on the venture
Proof of investing a substantial amount of own capital to bootstrap
Hard costs invested in development and tech, front end, graphic design, marketing, etc
Evidence of legal advice (expensive, and a sign of good risk management)
None of these are foolproof by themselves but applying this rule will steer you clear of most low effort rugs and save your valuable time.
Your Toolbox
Every non-technical user can access the following tools:
Analysing the project’s marketing
Evaluating the team
Assessing partnerships with other projects
Research governance forums, Discord, and Github
Peer review (including pre-built on-chain analysis)
An audit has some value for a non-technical reader as you can draw inferences from the reputation of the firm engaged and what they say in the summary section.
Type of Marketing
“If you can’t spot the sucker at the table, it’s you.”
While protocols using dodgy marketing techniques can rise in value (for a time), they are also more likely to be unsustainable or rugs. A red flag is “affinity fraud”, made famous by Bernie Madoff’s scheme:
Affinity fraud is a type of investment fraud in which a con artist targets members of an identifiable group based on things such as race, age, religion, etc. The fraudster either is or pretends to be, a member of the group.
In the context of social media, options for group identity are broader and more malleable than race, religion, gender, etc. Affinity fraud exists but is less obvious.
Creating a cult like following can be achieved by giving members a special title or rank with which to identify themselves, putting a logo / symbol in their social media handle, and so on.
How many phishing attacks and scam DMs have falsely claimed affiliation with the Bored Ape Yacht Club?
You shouldn’t penalize a project for strong marketing and creating a passionate community. But. ‘Lunatics’ and ‘Frog Nation’ are obvious examples of a charismatic leader and a cult of personality overriding common sense and sound judgment.
Autist note: Regular readers will know we’re financial history buffs: further reading is the 1841 study of crowd psychology by Scottish journalist Charles Mackay, first published as Memoirs of Extraordinary Popular Delusions
“Our team worked at FAANG” - assessing value of team
Like having attended a good university, this only demonstrates a minimum level of intelligence and willingness to “play the game” well enough to pass an interview. (@BowTiedFox explains how to game the FAANG interview process)
This metric is probably best ignored or given very little weight, unless a *senior* employee is leaving their corporate job to go risk-on in DeFi. Or they are very well respected in a technical / STEM niche e.g. particle physicist at CERN. Then it is an extreme positive (don’t bet against geniuses).
An anon who “worked in machine learning for FAANG” is probably not a good choice for managing your investment portfolio with their new trading robot.
More important than the employment history of the founders is who the protocol is employing, right now, to keep their operations secure. Ask! What information is available about the dev team? Do they use DevOps? Is there a SysOps person? Any ‘Red Teaming’ exercises or other *ongoing* security engagements with firms like Runtime Verification? Too often the team hire a frontend/backend dev and a solidity dev, audit one version of the code, and call it secure. Not good enough.
The founders may have the vision and the risk appetite but the day to day management of IT and other business functions can be critical to security. Ask about processes. BadgerDAO were warned of a possible exploit in Discord before they lost 9 figures of customer funds, but their moderator didn’t follow up.
9 figure DAOs ought to have professional management and formal processes to safeguard funds.
“We’re backed by Alameda” might mean the project is good, or it might mean that Alameda expects to make money providing market making services, or has some other motive such as buying a potential competitor or locking in talent via an acq-hire. Alameda-backed low float high FDV coins launched on Solana were among the worst performers last year.
Partnerships
“We use Chainlink”
They’ve avoided the likely trap of trying to implement their own oracle from scratch. Unless you’re MakerDAO you probably don’t have the expertise and budget to do this right.
So far, so good. But don’t assume that just because they’re associated with a trusted brand that the oracle system is secure!
Chainlink will let you point your oracle contract to a single web server which you control and from which you serve arbitrary data. This means that the project team (or an ordinary hacker who compromises a basic web server) could take over the oracle and report arbitrary or malicious values. So merely using a Chainlink adapter is not enough to guarantee security.
Instead, you need to dig into the implementation.
How many price feeds does it use?
Who controls these price feeds?
Chainlink can be configured to report the median (not mean!) value of multiple price sources which effectively excludes a minority of outliers or compromised feeds being able to affect the oracle price. If the contract is linked up to Coingecko, Binance, and the Uniswap pool for an asset it is unlikely that the protocol can be easily manipulated by an oracle attack.
If it is connected to a single private server…watch out!
Other partnerships
This isn’t a substitute for your own due diligence and is less useful for early-stage protocols (where most of the investment gains – and risk – can be found).
However, consider the following:
If a token is accepted as collateral on Aave or MakerDAO it is likely that experts have reviewed the code to make sure the token isn’t vulnerable to infinite mint attacks, doesn’t contain transfer taxes, etc. and that the admin key hasn’t been “lost” (actual event from a popular project).
If a protocol is covered by insurance you can infer something about their judgment on risk from the cost of cover, and also assume (if there is a high cover limit available, into the millions of dollars) that the insurance protocol team have done some diligence.
If the yield farm is being actively used in a Yearn strategy it has passed muster with them.
If there is a high-profile integration with a team / project you respect then it is likely that there has been some due diligence. For example, we would be shocked if there were obvious code quality issues at DeFi Kingdoms that weren’t picked up by the security chads at Synapse Bridge prior to their partnership. If @bantg is an advisor we wouldn’t expect sloppy code, and so on.
Governance Forums, Discord, and Github
Are the governance proposals reasonable? (six figure sinecures for random ‘advisors’ and treasury management proposals to invest in shitcoins = red flags)
How many wallets participate in voting? Is the protocol run by a centralized group of team/whales or is there broad engagement and participation across a diverse range of token holders of all sizes?
Do the team hold regular voice chats / Q&A in Discord? This has value because tough questions can be asked and the team cannot rely on a pre-prepared response.
Is there in-depth technical discussion? Some discernment is needed to distinguish genuine domain specific knowledge in the tech channels from made up marketing gobbledygook. If there’s lively discussion between smart techies this is a positive.
Look in Github for the users who check in code to the repository. Do they have a long history of participating and do they contribute to other projects? (instructions)
“But our project is Audited”
Very well. Three questions:
1. Is the auditor a respected firm or a no-name / fly by night outfit?
2. Were there any red flags in the audit, including unresolved issues?
3. Has the code been changed since the audit?
Not all auditors are equal.
Certora (formal verification), ConsenSys Diligence (from the makers of MetaMask/Infura), OpenZeppelin (authors of many popular standard libraries), and Trail of Bits are the best. Many “B-Tier” auditors are worth trusting.
The other auditors? Well, it’s your money. Refer to @BowTiedPickle’s ranking list.
Red Flags to watch for:
Summary section: any of the following a red flag - fewer than 2 weeks of effort, high severity flaws, dozens of low severity or informational issues
Scope: important protocol features being excluded from scope - you do need to understand what the protocol does to spot this
Code style / maturity / best practices: if there are positive things to say the auditor will (usually) say them. If this area seems critical of programming practices, even if you don’t understand the details, safest to avoid.
Recommendations / Issue Resolution: the auditor should confirm that any major issues found have been patched. Red flag if not.
Has the code changed?
The last one point is more difficult and might involve checking the chain, comparing documentation versions, or asking in Discord.
Basically, the rule of thumb is that if any major changes have been made to the code it should be re-audited, and you can’t consider any older audits as they refer to different code.
An example of a new feature requiring a new audit would be adding a staking contract (if it hasn’t been audited, there could be bugs which could allow a hacker to drain staked tokens or a mistake which prevents the tokens being withdrawn). If it is a lending platform or a DEX which adds a new collateral type or factory pool then this is unlikely to require a major change to the code.
Peer Review (including on-chain analysis)
A search of Twitter, Dune Analytics, and FlipSide Crypto will often reveal data analytics dashboards, wallet tracking, or pre-built data scrapes relevant to the protocol you are thinking about investing in. Basic on-chain analysis can be conducted using a block explorer like Etherscan.io without technical knowledge.
What do other (trusted) sources have to say? Search the Twitter accounts of @BowTiedNightOwl, @BowTiedIguana, @BowTiedBull, and @BowTiedPickle to see if the project has been mentioned. If you’re in any Discords with smart people you trust, search the project name or ask a question. Many people will gladly help you out, and criticisms are more likely to be true than endorsements / shilling.
We hope this is a good starting point for non-technical readers to judge security without relying only on an audit, and good actionable advice for everyone else.
This is a free post - if you found it valuable please give it a share!
If you want to browse our database of protocols where we give specific comments on software security, and much more, join our community of turbo autists today!
✓
Disclaimer: None of this is to be deemed legal or financial advice of any kind. These are opinions from an anonymous group of cartoon animals with Wall Street and Software backgrounds.
Thanks for the shoutout! I just published an article about how nontechnical users can read an audit report, and walk through a real example. https://bowtiedisland.com/how-to-read-a-smart-contract-audit-report/
Is the primary difference between affinity fraud and strong marketing that one rugs you and the other doesn't? How do you tell before the rug?
I would place the Jungle in 'strong marketing'. However, until the $Time rug, I cannot articulate why it falls into that category, but Frog Nation does not, based on my understanding of the criteria you laid out.
Premise 1: The Jungle is strong marketing (aka NOT affinity fraud)
Premise 2: affinity fraud criteria are 1) giving members special title or rank with which to identify themselves 2) putting a logo/symbol in their social media handle, 3) charismatic leader, and cult of personality
Test if the Jungle fits into the criteria listed:
1) giving members a special title or rank with which to identify themselves
turbo autists and/or bowtied
2) putting a logo / symbol in their social media handle
bowties in the handle, and in the PFP
3) charismatic leader and a cult of personality
Bull has charisma and strong marketing skills. The sticky phrases and themes could feed into cult of personality. TrollTuesday suggests power of personality.
Maybe there needs to be a 4th criterion: token from which they could profit. Even so, there's Synapse and JPEG'd. To emphasize, both are great projects that solve critical needs, but Bull and y'all are affiliated with them, so even that fails to distinguish.
Premise 1 contradicts Premise 2, so one of these must be incorrect. What aspect of the affinity fraud criteria that you laid out am I missing, or incorrectly applying? These criteria look easy to apply to projects I don't like, but along with the Jungle, this could be applied to the Dopex/Diamond Pepes and most other strong communities (and historically, Steve Jobs' Apple)
I believe in the Jungle, and think it's a strong positive, which is why your affinity fraud part confuses me. But members of Frog Nation would have said the same about Dani coins. What am I missing?