JSON is awesome because it provides an amazing way for you to send information from one place to another and be guaranteed that it can be decoded by whatever language is on the other end. It's also awesome because it's schema-less and can easy adapt to hold any data. It just sucks when someone starts entering it into a database and you need to get it out.
Please note that I'm not recommending you EVER put JSON into a database column. This is a quick fix but it's going to bite you in the butt later and someone is going to have to clean it up. :-)
One of the products I inherited has several JSON fields in a MySQL database that contain important information that we needed to created a report on. The end goal is to rewrite the sections of the code that read to and write from these columns but they're important and most of the site actually uses it (we found over a thousand places where it's referenced). As a quick fix we found mysqludf_json_extract which allows you to extract specific information from JSON encoded data in a database.
My favorite part about this project is it's description:
if you thought it was a good idea to put json blobs in mysql and are living with the consequences
It reminds me of this comment about regular expressions:
Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
I had to install the mysql header files (libmysqlclient-dev) and cmake before I got started:
sudo apt-get install cmake libmysqlclient-dev
mysqludf_json_extract hasn't been updated in a while (4 years when I wrote this) and it has a dependency(yajl) that has been updated recently. You need to make sure you build the 1.x branch.
git clone https://github.com/lloyd/yajl.git
cd yajl/
git checkout 1.x
cmake .
sudo make install
I also found that on Ubuntu you need to copy the library to /usr/lib for it to work correctly:
sudo cp /usr/local/lib/libyajl.so.1 /usr/lib/libyajl.so.1
Finally, when you install you need to specify MYSQL_HOME and MYSQL_CONFIG
git clone https://github.com/dkf/mysqludf_json_extract.git
cd mysqludf_json_extract
set MYSQL_HOME = /usr/lib/mysql/
set MYSQL_CONFIG = /usr/lib/mysql/
cmake .
make
make test
Again, also found that on Ubuntu you need to copy the library to /usr/lib for it to work correctly:
sudo cp src/libmysqludf_json_extract.so /usr/lib/mysql/plugin/
Finally, you need to setup the function in MySQL.
mysql> drop function json_extract;
Query OK, 0 rows affected (0.00 sec)
mysql> create function json_extract returns string soname 'libmysqludf_json_extract.so';
Query OK, 0 rows affected (0.00 sec)
Then you can use it just like any other function:
mysql> select json_extract("a", val) result from foo;
We've been using mysqludf_json_extract for several months now and the only real problem we've run into is if we try to perform a where on a field inside a JSON blob it can stop the query. We just told people it doesn't work that way. :-)
In closing, don't do this. It's a hung pain in the ass.
$ cmake .
CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly.
Missing variable is:
CMAKE_C_COMPILER_ENV_VAR
CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly.
Missing variable is:
CMAKE_C_COMPILER
CMake Error: Could not find cmake module file:/home/swarren/yajl/CMakeFiles/CMakeCCompiler.cmake
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!make:
sudo apt-get install makeI didn't have this extreme of a problem but I did have some performance problems with my Symfony project. With the other problems I've run into with shared folders recently I've started debated throwing out the whole thing and just using Vagrant for the deployment and using SMB from the VM.
The graph at the end of this post is great. I think I've seen a lot of Reckless/Deliberate projects when the original programmer didn't know what they were doing. :-)
We're using PHPUnit to add integration and unit tests at work and normally when I want to test a single I use --filter to pick just the one I want:
vagrant@precise64:/var/www$ vendor/bin/phpunit -c tests --filter UserTest::testLoginWithInvalidPassword
PHPUnit 3.7.31 by Sebastian Bergmann.
Configuration read from /var/www/tests/phpunit.xml
.......
Time: 6.82 seconds, Memory: 17.25Mb
OK (7 tests, 7 assertions)
Almost 7 seconds for 7 tests is a lot...
When I just just --filter PHPUnit is loading all the other unit test classes and then running just the ones that match the --filter. By specifiying the file I want to run the test from I can get better performance:
vagrant@precise64:/var/www$ vendor/bin/phpunit -c tests --filter UserTest::testLoginWithInvalidPassword tests/models/UserTest.php
PHPUnit 3.7.31 by Sebastian Bergmann.
Configuration read from /var/www/tests/phpunit.xml
.......
Time: 4.84 seconds, Memory: 12.50Mb
OK (7 tests, 7 assertions)
I'm sure for any PHPUnit masters this is old information but I've never seen it so I thought others might find it interesting.
Open plans have been surprisingly hard to kill, despite research showing that they’re unpopular, decrease employee satisfaction, and hurt productivity. The response so far seems to have been to double down and make it, if anything, worse: cubicles are now decidedly un-cool so no-wall open offices are all the rage, and Facebook brags that its new building will be the largest open floor plan in the world, consisting of a single, ten acre open room.
I've worked in a cube farm and a room where they jammed in six desks and computers. Working in an open area can be super annoying when the person next to you is loud, messy, and/or smells.
I'm most productive working from home and highly recommend people who can do work remotely.
http://blog.stackoverflow.com/2015/01/why-we-still-believe-in-private-offices/
Similarly, it is important to know when you are writing bad software. Here’s a few questions that will help us diagnose if you are writing bad software.
- Does pushing updates to the software take a lot of time and effort?
- Does the whole system go down when you push a very small change?
- Have you ever pushed broken code to production, and didn’t realize until your users started complaining?
- Do you know what exactly to do when your system goes down — how to dig into backups, and deploy them?
- Are you spending more time on things like moving between environments, or running the same commands again and again, or running small utilities than actually making the software?
If you answered yes to any of these questions, this post is for you. Read all the way through, at least twice.
https://medium.com/@karan/the-7-rules-for-writing-software-that-wont-die-when-you-do-38ef0a925650
I was trying to install compass on one of my VMs the other day and I ran into this error:
[vagrant@unknown080027e8e390 public]$ sudo gem install compass
Building native extensions. This could take a while...
ERROR: Error installing compass:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/ruby.h
Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/ffi-1.9.6 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/ffi-1.9.6/ext/ffi_c/gem_make.out
The quickfix for this is to install the ruby development package:
sudo yum -y install ruby-devel
I'm not sure why CentOS doesn't install this with rubygems...
We had a developer quit last week so I did the most logical thing, I started deleting their access to our systems. Website login - check, QuickBooks login - check, email - check, GitHub - check. Then I started working on trying to finish some of the pull requests that he had outstanding.
subscribe via RSS