Hi Shawn, good questions. I have a 4yr old and a 9mo old and we spend a good amount of time in SLU and want to live there, but haven't found the right place yet. According to population estimates at sites like Redfin, there are already almost 200 children living in SLU (51 of them under 4yrs)--I've only seen them a few at a time though. :)
I don't know anything about your family's preferences but here are some things we like: very close to my work, 2 parks with playgrounds, Farmer's Market, Whole Foods across the street, indoor play area at REI for those cold rainy days, close to Seattle Center museums
The only safety concerns I have had is crossing busy streets like Denny or Westlake. I always look both ways twice and still have had close calls because there are so many drivers who only see SLU as a place to speed through.