Since the Kickstarter campaign, we have been working hard on producing the product that was planned and prototyped. It has been a busy few months and we are happy to see many of the functions falling into place. As some of you have noticed, there are still areas that requires improvements. One such area is motion detection. Read about what is happening and the plan ahead.
We have now shipped thousands of Points. For many it works just like it should, but for some, Point is not picking up movement when it should. There is a variation between units that is bigger than expected and was unknown prior to shipping.
During discussions with early users and learnings from our first generation of Point users, we decided to make the alarm less sensitive. We believe that false positives have a significantly worse impact on the user experience compared to missing an event.
That being said, what are we doing to improve sensitivity on the devices that are affected? A few things. First, lets go over the development and test cycle, then let’s look forward a bit on additional aspects.
Our in-house test fleet contains roughly 100 devices located in a controlled environment. We deploy all software updates across this fleet and monitor it carefully for errors. Each test cycle is at least 48 hours, but for testing movement in particular, we prefer to run each test for up to two weeks. 100 devices running for two weeks results in almost 4 years of running a single unit.
There are quite a few knobs that affects sensitivity. All of these are controlled through software. The first step we took was to find a baseline where no devices in our test fleet generated any false positives. Remember that we consider these worse than false negatives. From there, we continued to gradually increase sensitivity.
After we are confident the devices perform well in a controlled environment, we deploy the updates to a smaller fleet installed in the homes of team members. This allows us to get first-hand experience of the product and we all use it daily in our lives. For those of you familiar with the process, this is called dogfooding. It’s not a fault-free process however, and across the devices we had in our homes, none of us experienced the lack of sensitivity. We are taking steps to improve this phase of testing in the future, such as ensuring we have a broader variation amongst the devices and install configurations.
The plan ahead
We are continuously iterating on the parameters to increase sensitivity without letting false positives slip through. The latest run has been going for little over a week and is already looking better.
Firmware updates will continue to happen in the background, but we will start publishing changes with each release in our Help Centre. When a new version is available, we will also post in the Point User group on Facebook and update our changelog as requested by many of you.
We are exploring many more ways to improve the detection of movement. There are possibilities to improve the feature using other sensors in Point, further calibration, and learning from the data collected. We can’t provide a timeline on specifics, but will continue to share updates as improvements are rolled out.
Automatic alarm toggle
We set out to create a home alarm that is easy to use. A part of this means not worrying about forgetting to turn on the alarm as you leave home. This feature is now being tested internally and we’re looking forward to start sharing it with a broader group as soon as we’ve squashed a few remaining bugs. This function will undergo a developing process as well.
You are championing a new type of home alarm. We are thrilled to have you onboard. We read every email, and every comment on Kickstarter and in the Point User group. Thanks for your patience as we continue this journey.
Everyone at Minut