You found my old blog. Thanks for visiting! For my new writing, visit mikesententia.com.
In a car, the pedals are the interface, and engine is the implementation. Here’s how to design tests to explore magick’s engine.
This series is about two things:
- Why my model of magick is useful.
- How I know it’s accurate.
To answer those, I’m going to show you each part, what it does, what it predicts, and non-obvious techniques I’ve developed based on those predictions.
But first, we need a few concepts. My model focuses on magick’s implementation, rather than its interface. And my testing focuses on predicting new techniques, rather than A / B testing. Those aren’t standard terms from other forms of magick, let’s cover them before diving into the details of my model.
If you’re new here, read this quick intro before reading this series.
Interfaces and Implementations
Interfaces are about information and instructions. A computer’s interface — the mouse and keyboard — is about telling the computer what to do. Same with a car’s interface: The pedals and steering wheel. Interfaces are where users normally focus.
Implementations are what’s going on under the hood. A computer’s implementation is transistors, drivers and code. A car’s implementation is the engine, rack-and-pinion steering, and disk brakes. Implementations are normally hidden from users. It’s where engineers focus.
In magick, the interface is how you signal your intent: Rituals, visualizations, and other uses of symbols. The implementation is, well, whatever happens externally to produce the results. That’s the focus of this series.
What about testing rituals to uncover “natural laws,” like the effects of certain symbols? Aren’t natural laws fundamental, irreducible truths?
Usually, natural laws describe a particular interface. But if you work exclusively with one interface, it’s easy to forget there’s an engine behind it, doing the actual work.
Think about it this way: If you’d never heard of a car engine, but someone told you that the pedals operate by natural laws, you’d test the relationship between foot pressure and speed, accelerating in different road conditions, and if it’s better to press slowly or quickly. Then you might try those tests on different cars, and trains and jet skis. But it would take a real leap to step outside the car, pop the hood and examine something that’s intentionally hidden.
I don’t want to put down anyone who experiments with rituals. Mastering an interface is important. I use three systems (the forces mages channel) almost every day, and I’m glad to know which commands they respond to. But I’m hoping you also want to understand how those instructions become changes in the world, both out of human curiosity, and so you can build better techniques. Because that’s my passion, and that’s why I focus on magick’s implementation.
You don’t find magick’s implementation by experimenting on rituals any more than you figure out engines by testing pedals, or learn to build a motherboard by using software. No, you need to pop the hood. That’s what we’ll do in this series.
Here’s more on interfaces and implementations. For more on “natural laws,” see my favorite philosophy blog, Less Wrong. “Natural law” is a semantic stopsign similar to “elan vital.”
Scientific Models
A good scientific model has two properties:
- It predicts a specific, testable outcome.
- That outcome is non-obvious, and not predicted by other models.
A model is like a map, and the only way to see if the map matches the territory is to ask it what will happen in a bunch of specific situations. Because in order to be correct in all those situations, a model needs to closely align to how the world actually works.
Imagine a weather model that reliably predicts the exact minute the rain will start. The only way it could do that is by matching the way rain actually happens.
But imagine a model that says “It will rain sometime in April.” Sure, the prediction was correct, but it was also obvious. Any model can make correct, vague predictions, even if it’s totally out of sync with reality. That’s why we don’t give any points for obvious answers.
And a model that makes every outcome equally likely, like “Whatever you believe will happen, happens”? Well, that’s not really a model at all.
Testing Magick Models
I realized I need to discuss different testing methods after reading Ananael’s comment:
I can say “I hypothesize that set A of ritual forms should produce a higher probability shift than set B” without too much difficulty, but […] I need [to do] several hundred of those [rituals] for my sample.
I’ll call this “A/B testing” (a marketing term), because you’re testing between two specific options (A and B). It’s a form of “black box testing,” which means you don’t need to understand the inner-workings of the thing you’re testing, you just try it and see what happens.
To do A/B testing, you need to be able to enumerate the options. This is fine for rituals, where you’re deciding whether to do a pentagon or hexagon, and in which direction. It’s a problem for exploring magick’s implementation, where you can’t even list all the moving parts at first.
The real problem is that black box testing doesn’t include an actual model. There are no moving parts, so you can’t think through what would happen in different situations. All it can tell you is whether A works better than B, not why, and not what would happen in situations C or D.
I don’t want to single Ananael out. I’m sure he does a variety of tests. I just used his comment because it got me thinking about this.
In other words, A/B testing is a great tool if you want to refine a ritual, but it’s the wrong tool for understanding the inner-workings of magick.
Instead, I mostly rely on direct observation and technique prediction:
Direct observation means using sensory connections to watch mental muscles and systems as they work. Like how dissection was the first step to modern biology and observing planetary motion was the first step to modern physics, direct observation is the start of understanding magick’s implementation.
Just like observing planets accurately requires a telescope and observing cells requires staining, directly observing magick requires its own technique: sensory connections. They let you see energy signatures and the structures that energy flows through, including mental muscles, energy paths through the body, and everything else that systems do to implement magick. We’ll discuss them later in this series. For now, just know that there is a (moderately advanced) technique to observing magick accurately.
One thing I want to make clear up front: Direct observation is not closing your eyes and visualizing how magick works. Those visualizations come from your already-existing expectations, not the external world. They’re guesses, not observations.
How do I know my observations are accurate? Because they (and the models they suggest) predict techniques. This is similar to A/B testing, in that you try a technique you haven’t done before and see how it works. But instead of black-box testing, where you guess at what changes to make, this is white-box: You have a model of the world, simulate it in your thoughts, and see what you should do to accomplish some task. If your technique produces the expected outcome, that suggests your model is on the right track*, especially if no other model would have expected that technique to be effective.
*A model can never be correct. “The right track” is all we can aspire to. Like Newtonian physics, to be replaced by Einstein 300 years later.
But wait. If I expect X to work, and then I see X work, how do you know my results aren’t just “Whatever I expect to happen, happens”? Simple: Because X doesn’t always work. I have a pretty good success rate, because my predictions are based on direct observations. But it’s far from perfect. And my confidence has little to do with the results: Sometimes I’ll be pretty sure of a technique, and it fails. And sometimes I’ll just try something to see what happens, and it works. The result is in the actual technique, not in my expectations.
More on how I avoid coincidence, placebo and more coincidence in my testing.
Comments
See a flaw in any of that? Leave a comment. I’m always looking to improve my methods.
Next
With those terms taken care of, we’re ready for details and experiments with my model.
If you liked this post, consider visiting my current blog at mikesententia.com.
I disagree completely that A/B testing is the wrong tool for understanding the inner workings of magick. Because it’s important whether or not you’re working from a model based on direct observation. Let’s say that you’ve developed your model and it suggests a technique. I assume you’re not just trying out this new technique and seeing if “it works.” You need to compare it to an existing technique or techniques and then see if the statistical probability shift is higher for the new technique than the old one – that is, A/B testing.
If you don’t take that step I would contend that you’re not learning much from the fact that you can use a new method to get some results – as we all know, magical methods vary widely and most of them have survived over time because they work to some degree. If you’re trying to optimize your magick, the goal should be to develop a model that suggests more effective techniques, not just new ones that are about as effective as the traditional ones.
I think that’s fair. Thanks for bearing with me, I’m still figuring out how to explain all this.
My real point isn’t about testing 2 options vs 3 or 5. It’s about how to come up with the options to test. I may have misread your previous comment, but the impression I came away with was “I’ll test if a clockwise pentagram works better than a counter-clockwise one, so then I’ll know which one works better.” As opposed to “Given these inner-workings of magick, if I do X, this result should happen.”
Sometimes, I’m trying to do something faster. Heal in minutes instead of hours, say. In those cases, you’re right, the old way was my A case, and the new way is my B, though I haven’t thought of it in those terms before.
Sometimes, the new techniques I make are for problems I haven’t solved before. It’s not coming up with a way to get a bigger probability shift, it’s coming up with a way to heal a different type of injury, accurately communicate letters, bypass a new type of shielding, things like that. So I don’t have an A to test against (or it’s the implicit A of “Do something and get no results”). In those cases, I consider a success as a confirmation of the model.
Does that seem fair to you? I’m genuinely asking, because it’s always good to get a neutral opinion on these things.
In my previous comment I was talking about the testing phase of the scientific process. This is distinct from the hypothesizing phase. From how I read your post here, it sounded to me like you were conflating the two. If I were trying to test out pentagrams, for example, I wouldn’t do it at all unless I had some reason to think that the system I was using was less effective than it could be, based on either observation or a model. The operant field model was developed in part because I experimentally discovered that it seemed the LBRP/LIRH was a more powerful ritual opening than the LBRP/LBRH. I started out with using it as a daily practice and I just generally felt better overall. That’s phase one – observation. Phase two is formulating a hypothesis, which in this case was “magical rituals opened with the LBRP/LIRH will produce bigger probability shifts than those opened with the LBRP/LBRH.” Phase three is testing, which I performed by doing a substantial series of rituals in which the only variable I manipulated was the opening procedure, and indeed found that the LBRP/LIRH was more effective to a statistically significant degree.
A hypothesis can also be formulated based on the predictions of a model rather than a direct observation, which is what it sounds like you do most of the time. That’s fine and I do it as well to some extent, but you do have to be careful when doing that sort of work. If you really want to be scientific, you do need some sort of a control group. So instead of testing your model’s prediction and seeing if you get some sort of result you need to compare it statistically against something that the model does NOT predict. Falsifiability is vital to the scientific process, and it gets missed a lot. For example, Freudian psychology was developed by trying things and seeing if they worked on patients, and the model Freud developed from those observations is, as philosopher of science Karl Popper once pointed out, useless in a scientific sense.
If your A/B is get results/get nothing you need to have a control group in which you do nothing. In other words, you should identify a series of recurring events that you want to influence, then determine the outcome you want and cast spells at those events half of the time. The other half of the time you do nothing – this is your control group. Then you compare the likelihood of success in the two groups. If they’re the same, your spell didn’t do anything even if some of the time your trials succeeded.
I was trained in experimental psychology, which is the most paranoid science in the world as far as sticking to the scientific method goes. Does it show? ;-) As a caveat to all of the above, I do realize that these conditions are often hard to set up when dealing with a phenomenon such as magick, and the above represents more of an ideal than what I go through every time I want to test a new technique. Still, without that level of testing you can’t be as sure as it sounds like you want to be that your model is the best description out there. There’s also the issue of repeatability, which is the biggest sticking point with magick’s skeptics and also means that even if you figure out a way to, say, heal someone in minutes when it used to take you hours, in order to make the claim that your model represents a general solution you would have to establish that either (A) no other healers can work that fast or (B) those that can are using some variation of your model.
That’s kind of where I’m at with my operant field model at this point. So far every magician I’ve know personally who has tried it gets better results, but it’s still a group that’s too statistically small for me to go out and start making general claims about how all magick works.
Hey Ananael, you make some really good points. I’m realizing that it’s not just a matter of explaining myself poorly, it’s that I haven’t fully developed my intuition on testing into ideas I can write down. So I’ve been figuring it out more, and should have something later this week.
Thanks for taking the time to write the comments.
My first attempt at explaining those intuitions and developing a useful philosophy of testing:
https://magickofthought.com/2011/11/8-new-ideas-on-testing-magick/